I once found myself sitting in an active war zone without my gear. It had been delayed and held up for reasons not worth getting into, even though the logistics lead confidently informed me that the shipping conduit we were using was "on average 3 days faster" than the standard cargo route.
Maybe he was right, maybe he was full of sh*t. None of that mattered when I arrived with most of my team and our stuff wasn't waiting for us like he assured us it would be. For those without a military background, this might sound strange. Doesn't everyone just board the same plane with your gear and fly there? Yes, that's often how it works. But there are times when that's not the case, for a variety of reasons.
What matters for this story is that we shipped via a route that should have been "faster on average." Maybe it actually was. The issue is that it wasn't faster this time. What mattered wasn't the average; it was the variance around that average. What mattered was consistency and predictability. What mattered was reliability.
If one method takes an average of 3 days, plus or minus 2 days, it could take 1 or 5 days. If another method takes 4 days exactly, without fail, it's slower on average, but it's reliable and predictable. You can plan a mission much better around reliability and predictability.
Most operators running businesses in the sub-$50M revenue space understand this concept intuitively. They've dealt with the frustration of unpredictability with employees, vendors, and even customers.
Today I want to make the case that variance itself, for a handful of key metrics, is something worth measuring and tracking on its own.
The Vendor You Actually Fire
Think about the last vendor you fired. I'd wager it wasn't the slowest one, and it probably wasn't the most expensive one either. A slow vendor can be planned around, as long as they deliver a quality product or service. An expensive vendor can be negotiated. The vendor that gets replaced is the one you can’t predict: great last month, a mess this month, and no way to know which version will show up next.
The reason is that inconsistency charges an operational tax that slowness and cost don't.
Every unpredictable interaction forces you to re-verify. You double-check the order. You follow up on the ticket. You keep your own copy of the numbers. The vendor may be billing you for the service, but the re-verification is the real price, and you pay for it in attention.
Here's the uncomfortable part: your customers experience your company exactly the same way. And most companies can't see it, because everything they measure is an average.
Average resolution time, average NPS, average project margin. The customer doesn't live in the average. The customer lives in the spread: which technician showed up, which rep answered, whether the invoice looked like last month's. When they leave, the stated reason will be price or "a change in direction." The actual reason, more often than operators want to believe, is that dealing with you required too much of their attention.
AI Cuts Both Ways Here
The most common objection to putting AI in front of customers is that it adds variance: the same AI agent can produce different outputs, and the model can change underneath you between Tuesday and Thursday. That's true of the raw model, and an ungoverned deployment can absolutely raise customer-facing variance while the pilot deck reports success based on averages.
But nondeterminism is a property of the raw model, not of a well-constrained system. A well-built agent gives consistent answers on Saturday night and Tuesday morning. It doesn't have a rough week, doesn't get angry with a customer, and doesn't depend on which human happened to pick up. For a services business, I'd argue consistency is the most underrated part of the case for AI. More than speed. More than cost. The determining variable is whether the deployment treated variance as a key spec or not.
Measure The Spread
This is the move an operator can make: for your most important KPI, promote variance to a first-class metric. Pick the one customer-facing process that matters most and start reporting its variance alongside its average. Not just "what's our average response time," but "what's the spread, and what was the worst interaction a customer had with us this month." What gets measured as a residual stays a residual.
One more observation, which I'll expand on next week: when you find that variance, it's usually not a training problem. It's a consequence of how the company was put together.
In the meantime, one question worth sitting with:
How much does your customer's experience of your company vary depending who, when, and where they interact with it?
The logistics lead had an average. My team needed a specific date.