AI Agent Token Spend Is Random. Governance Has to Match

Gartner predicts over 40% of agentic AI projects will be canceled by the end of 2027, citing escalating cost. The root cause isn't a bad model — it's that nobody can predict what an agent will spend before it runs.

Why agent spend can't be estimated in advance

Token generation is stochastic: a model samples its next token from a probability distribution, so the same prompt can return a different completion — and a different token count — on every run. OpenAI states its APIs are "nondeterministic by default, and that determinism is not guaranteed even with a fixed seed." An agent picks its next action from that unpredictable output, so the number of reasoning steps and tool calls it takes, and therefore the total spend, is a random variable rather than a fixed cost.

Agents also multiply that baseline cost. Anthropic has measured agents using roughly 4x the tokens of a single chat exchange, and multi-agent systems about 15x — with token usage alone explaining roughly 80% of the variance in total cost.

The three failure modes

A stuck recursive loop is the clearest one: an agent calls a tool, gets a weak result, feeds its own output back in, and calls the same tool again — fifty times, if nothing tells it to stop — while still returning a clean-looking summary that shows success on every functional dashboard. The spend shows up in exactly one place: the bill.

A quieter leak comes from cache busting. Repeated input can be cached at a steep discount — OpenAI documents 50% off cached input, and Anthropic charges roughly 90% below base input for cache reads — but that discount only applies to the prompt prefix that stays identical. Pasting a timestamp or session ID near the top of a system prompt breaks the cache for everything below it, so a large context that should cost a tenth of full price pays full price every turn.

A third leak is reasoning bloat: reasoning models spend hidden tokens before emitting any visible output, and those tokens are billed. A runaway reasoning loop can burn an entire budget inside one call without returning a character, and a breaker that only watches the visible output stream is blind while that happens.

Why provider guardrails don't catch it

Existing provider tools were built for alerts, not enforcement — OpenAI's own project budgets are described as "alerts, not hard caps." That leaves a gap between watching spend after the fact and actually stopping a run before it crosses a limit, which is the distinction between observing token cost and governing it.

What to check this week

If your team runs multi-agent workflows in production, check whether any agent's control loop can currently stop a run mid-execution on a spend ceiling, or whether your only signal is the monthly invoice. Also check where volatile values like timestamps or session IDs sit in your system prompts — moving them to the end of the prompt, after static content like tool definitions, is a one-line fix for the cache-busting leak.