Why your AI bill keeps growing while token prices fall: A CIO’s guide

Key Takeaways

• Paying less per token has not made AI cheaper. Usage is growing far faster than prices are falling, so total spend keeps climbing. 

• The bill is set by how systems are built, not by what the vendor charges. Agents use roughly four times the tokens of a chat interaction, multi-agent systems around fifteen times, and much of it goes into redoing work already done. 

• So this is a design problem, not a purchasing problem. You cannot negotiate your way out of it. What changes the bill is where reasoning happens and what governs it. 

“Our conversations are never about [what it can do] now. Now the conversations are about, ‘hey, we’re spending so much. What visibility do you have? What auditability do you have? What token controls do you have?’” – Alexander Embiricos, OpenAI, speaking to TechCrunch 

That is how the questions from the largest AI customers have changed, and we see the same shift with our customers too. At the Chief AI Officer Exchange in Chicago in May 2026, cost governance came up in conversation after conversation, across industries. Two years ago, the question was what the models could do. Now it is what they cost to run. 

Token prices are falling, and your bill is still rising because the price per token was never what drove the bill. Consumption is. And consumption is set by design decisions: how work is scoped, what context gets loaded, how many times an agent revisits a problem it has already solved. 

Won’t cheaper tokens fix AI costs on their own?

No. Economists have a name for this: Jevons Paradox. When something valuable gets cheaper, consumption rises faster than the rate at price fall. It happened with coal, with electricity, and more recently, with bandwidth. Now, it is happening with AI.  

At Google I/O in May 2026, Sundar Pichai said Google now processes over 3.2 quadrillion tokens a month, more than 300 times the 9.7 trillion it processed in 2024. Then he added: “We’ve heard that many companies are already blowing through their annual token budgets, and it’s only May.”  

Using more isn’t the problem, though. Paying twice for the same thing is – in this case, the thinking.   

Cognitive redundancy: Where the money actually goes

Cognitive redundancy is what happens when a system redoes thinking it has already done and gets billed again for reaching the same answer. 

Take a change request moving through a software lifecycle. An agent reads the ticket, loads the repository context, plans the change, writes it, runs the tests, reads the failures, reloads the same context, and tries again. Every pass is billed. By the third attempt it is paying to rediscover what it already knew on the first. 

Anthropic’s engineering team measured what that costs: agents use roughly four times more tokens than a chat interaction, and multi-agent systems around fifteen times more. The same task can cost wildly different amounts from one run to the next, which is the same non-determinism that makes these systems hard to test. 

“The expensive part is almost never the first answer. It is the third and fourth attempt at the same answer, because nothing in the system remembers what the first two established.” 
– Ritesh Kumar Singh, Delivery Manager, Zuci Systems 

Why do AI costs go unchecked?

Because three common assumptions about AI economics keep them in place, and the data contradicts all three.

So the invoice ends up doing the job a monitoring system should be doing, and the problem is always found after the money is gone. The culture even has a name for the third assumption, tokenmaxxing: treating heavy usage as a sign of engagement, and rewarding it rather than questioning it. 

Many organizations are also waiting for a better model to absorb the redundancy. It will not. A better model does not redesign your workflow. 

Execution architecture: How do you stop paying for the same reasoning twice?

Better prompts and cheaper models don’t solve cognitive redundancy. We need to change the execution architecture : more specifically, where reasoning happens, and what governs it. 

Enterprise systems used to have clean boundaries: decision support recommended, workflow engines executed, rules engines enforced policy, people provided oversight. AI collapses all of it into one continuous reasoning process. Judgment, execution, and validation now happen inside a single probabilistic step. 

Two design decisions change that.   

1. Separate cognition from execution

Cognition handles what needs judgment: ambiguity, conflicting signals, situations the system has not seen before. Execution handles what must be identical every time: orchestration, validation, policy enforcement, audit logging.  

For instance, in fraud detection, pattern detection and risk assessment belong in cognition; transaction processing and audit trails belong in execution. Splitting the two turns a single AI system into a Hybrid Intelligence System: AI reasoning, deterministic controls, and human judgment, each doing what it is best at.  

“We start by asking which steps genuinely need judgment. It is almost always fewer than the design assumes,” says Feroz A, Solution Architect at Zuci Systems. 

2. Make governance part of execution

Separating cognition from execution cuts out the reasoning you never needed. It doesn’t control the reasoning you keep. 

Left unchecked, an agent can retry a failing task again and again, reload the same context, switch to a pricier model, or keep reasoning long after the answer has stopped improving. Every one of those attempts consumes intelligence, and you pay for it whether or not the outcome gets better. 

That’s why Governance by Design has grown beyond a compliance requirement, into an economic necessity today. The rules have to be built into execution: limits on reasoning and retries, clear conditions for escalation, a quality bar every outcome has to clear, rules on which model each step can use, and a point where a person steps in when the system can’t make progress. 

The aim is to spend intelligence deliberately, within limits, and in proportion to what the task is worth. 

And you can only govern what you can see. A Continuous Intelligence Control Plane shows where intelligence is being consumed, flags wasteful execution patterns, enforces the limits, and keeps tuning performance and cost. Most teams put off building one until scale forces the issue. 

We used to treat runaway AI cost as a prompting problem. A prompt does not change where the work happens. Moving the deterministic steps into code did, and the bill stopped surprising us.” 
– Feroz A, Solution Architect, Zuci Systems 

Cognitive FinOps: How do you keep AI spend under control?

Cloud hit the same wall: unmanaged scale proved economically unsustainable, and FinOps emerged because the architecture underneath was measurable. AI is at that same point now. 

Structuring execution is what makes the spend measurable. When reasoning happens in discrete steps you can attribute, you can see which workflow consumed what and decide how much reasoning it is worth. That is Cognitive FinOps: the discipline of optimizing the business value generated per unit of intelligence consumed.  

In practice it looks like cloud tiering. You don’t run every workload on your most expensive tier. Classification needs a lightweight model; genuinely ambiguous analysis justifies frontier reasoning. Most enterprise work is not the second kind. “Choosing a model should be as boring a decision as choosing a server tier. Right now it is mostly made by default,” Ritesh points out. 

Each of these helps on its own. The bigger payoff comes when they work as one system, which we call Enterprise Intelligence Infrastructure. It rests on five capabilities: 

Here’s how they complement each other in the fraud detection example: Execution architecture keeps transaction processing and audit trails in deterministic code. The hybrid design sends only the ambiguous patterns to AI, and the hardest calls to an analyst. Governance by design caps retries, sets which model each step may use, and defines when a case escalates. The control plane shows which cases consumed the most reasoning and whether that reasoning changed the outcome. Cognitive FinOps turns all of it into a number the business can manage, such as cost per case resolved. Take any one away and the rest weaken: without the control plane, there’s nothing to measure, and without the architecture, there’s nothing to govern. 

What should CIOs do about AI cost? 

AI cost is not a pricing problem you can negotiate. It is an architecture problem you have to design for. 

Start by mapping where reasoning happens today. Find the workflows re-deciding what they have already decided, not just the token totals. You cannot redesign a cost center you have not located. 

Assign an owner, the way you would for cloud spend. If nobody is accountable for it, nobody is ever questioned about it. 

And treat this as an architecture decision, not a procurement one. Favorable vendor pricing will not last. If your AI economics only work while someone else absorbs part of the cost, that gets exposed the moment pricing shifts, and no negotiation fixes an architecture that was never designed to be efficient. 

Our whitepaper sets out the architecture in full: Why Enterprise AI Needs Execution Architecture, Not Just Bigger Models.