Why AI FinOps Will Become Critical for Enterprise AI
Every cost discipline that has ever failed in IT has the same shape — the people who can change the number never see it. Enterprise AI reproduces that shape almost perfectly, which is why this becomes urgent sooner than most organisations expect.
Cloud FinOps grew up around a reliable assumption: cost follows capacity. You provision a virtual machine, you pay for the virtual machine, and the unit of account is time. Everything downstream — tagging, showback, rightsizing, reservations — rests on that one idea.
Enterprise AI breaks the assumption in a specific way, and most cost governance built for the cloud does not notice it happening.
The unit of account moves
In classic cloud, spend attaches to a resource. Tag the resource group, and you know which team owns the bill.
An Azure OpenAI deployment is also a resource, and it will also carry your tags. But the tag tells you that the deployment cost forty thousand euros last quarter. It does not tell you which assistant spent it, which department asked the questions, or whether the money went on answering people or on retrieving documents nobody read.
The meaningful unit is no longer the resource. It is the request — and inside the request, the tokens. That number is produced by application behaviour, not by an infrastructure decision, and nothing in your existing cost model is looking there.
Cost is now set in application code
This is the part that catches finance teams off guard.
A developer improves answer quality by passing more retrieved context to the model — say the retrieval step returns eight passages instead of four. Input tokens roughly double for every request that path serves. It is a sensible engineering change, made for a good reason, and it is arithmetic rather than a prediction: twice the context is twice the input.
Now notice what did not happen. No larger VM. No new resource. No procurement conversation. No approval gate, no architecture review, nothing a budget alert would fire on. A one-line change to a retrieval parameter altered the marginal cost of the entire service, and the person who made it may not know a cost model exists.
Traditional cloud controls are calibrated for a world in which spending money required deploying something. In AI, spending money requires a merge.
Success makes it worse
The second surprise is directional.
Almost every other enterprise system gets cheaper per user as it grows. Fixed costs amortise, licences tier, the platform is already running. Roll it out more widely and the unit economics improve. That instinct is so deeply held that nobody states it out loud.
Inference does not work like that. Marginal cost per conversation stays broadly flat — commitments and caching bend it, they do not break it. Double the adoption and you roughly double the spend. The pilot that cost a few hundred euros a month was not cheap because it was efficient. It was cheap because twenty people used it.
So the most dangerous moment in an AI programme’s budget is the one everybody is celebrating: the point where the assistant becomes genuinely useful and usage climbs.
Agents widen the distribution
A retrieval-and-answer service has a predictable cost per request. An agent runs until it decides it is done, so a single user request might be two model calls or fifteen.
The practical consequence for budgeting is that the mean stops being the useful statistic. Capacity and cost need planning against the tail — p95 calls per task, with a hard iteration cap so a pathological request cannot run all afternoon. Teams that forecast on averages find the variance in production, at scale, in the same month adoption climbs.
The old levers do not fit
Rightsizing has no analogue here. You cannot rightsize a conversation. The levers that actually move AI spend are different ones:
- Context discipline. Retrieve fewer, better passages. Most retrieval pipelines send more context than the answer requires, because more felt safer during development.
- Model routing. Classification, extraction and routing rarely need your most capable model. Sending every step to the largest one is the most common avoidable cost in production AI.
- Caching. Prompt caching for stable system context, and caching answers to questions a workforce asks repeatedly — which, in an enterprise, is most of them.
- Output limits. Verbose answers cost more and are rarely better. Cap them deliberately rather than accidentally.
- Commitment models. Provisioned throughput trades flexibility for a lower effective rate. It is the analogue of a reservation, but the risk profile is different, because the demand you are committing against is behavioural rather than architectural.
Each of these is an engineering change. None is a procurement decision. That is the whole point.
Cost and quality have to be measured together
There is an obvious failure mode in everything above: all of these levers reduce cost by reducing how much the model is given or allowed to do. Applied without measurement, that is not optimisation. It is quality degradation you cannot see.
Which is why AI FinOps and AI evaluation are the same programme wearing different badges. A fixed evaluation set — questions with known-good answers — is what lets you say that the cheaper configuration is also an acceptable one. Without it, every cost saving is a gamble settled later by user complaints, and the team learns to stop optimising.
The number executives will eventually ask for
Sooner or later somebody senior asks what the assistant costs per conversation, and whether that is less than what it replaced.
It is the right question, and most organisations cannot answer it. Not because the data is unavailable, but because nobody instrumented the request: which use case, which model, how many input and output tokens, how many calls. Capture those four things per request from the first deployment and the answer is a query. Retrofit them after a year of production and it is a project.
The comparison matters more than the absolute figure. A cost per resolved query is a number a business can weigh against the cost of the process it displaced. A monthly Azure OpenAI total is a number nobody can act on.
Why this becomes critical rather than merely useful
Look at the shape of the problem.
The spend lands somewhere no existing control is watching. It is set by people who never see the invoice. It grows in direct proportion to how successful the programme is. And the only way to reduce it — changing context, models, caching and limits — requires the engineers to know what their choices cost.
Every cost discipline that has ever failed in IT has that shape. The discipline is not really about the cost. It is about closing the loop: putting cost per use case in front of the team that can change it, in days rather than at quarter end, next to a quality measure that tells them whether they broke something to get there.
Do that while the numbers are small and it is a dashboard and a habit. Leave it until the numbers are large and it becomes a freeze, imposed by someone who could not evaluate the system any other way — which is how promising AI programmes quietly stop.