AI Got 280x Cheaper. So Why Is Your AI Bill Exploding?

Two things are true at once, and almost every AI budget was built on only the first one.

Per-token prices have collapsed. A query at GPT-3.5 quality cost $20 per million tokens in November 2022 and $0.07 by October 2024 (Stanford HAI AI Index). That's a 280-fold drop in under two years — one of the fastest cost declines in the history of computing.

And yet finance teams are watching AI line items double. Some engineering orgs burned through their annual AI coding budget before May (J.P. Morgan Private Bank, July 2026).

Both facts are real. Three separate mechanisms explain the gap, and the third one only started in December 2025 — which means the assumption that intelligence keeps getting cheaper may be wrong for the first time.

Key Takeaways

• Per-token cost at fixed quality fell roughly 280x from late 2022 to late 2024 (Stanford HAI).

• Agentic tasks consume around 1,000x more tokens than chat, with up to 30x cost variance between runs of the same task (Stanford Digital Economy Lab, May 2026).

• Token prices have risen more than 60% since December 2025 as compute ran short (J.P. Morgan, 2026).

• Cost is now an architecture decision, not a line item you review later.

Did AI Actually Get Cheaper? Yes, Dramatically

The decline is real and it's better documented than most AI claims. Epoch AI tracked state-of-the-art models across six benchmarks and found prices falling between 9x and 900x per year, depending on which capability level you hold constant (Epoch AI, March 2025).

That range needs unpacking, because it's the source of most confusion. You don't measure AI's price — you measure the price of a fixed level of ability, then watch it fall as cheaper models reach that bar.

Source: Epoch AI, 2025

Gartner expects the trend to continue on the supply side, predicting that by 2030, running inference on a one-trillion-parameter model will cost providers over 90% less than in 2025 (Gartner, March 2026).

Note the phrase "cost providers." What a model costs to run and what you get charged are different numbers. That gap is where this story lives.

So Why Are the Bills Going Up?

Because price per unit is only half of a bill. The other half is units consumed, and consumption has grown faster than price has fallen.

The clearest illustration comes from J.P. Morgan's analysis: a simple email draft uses under 1,000 tokens, while a specialized coding task can burn up to 56,000. As of June 2026, the top 1% of users consumed 600 times more tokens than the median user.

Same price list. Fifty to six-hundred-fold difference in the bill.

Three mechanisms are stacking, and they compound rather than substitute.

Mechanism 1: Reasoning Models Think Out Loud, and You Pay Per Word

Reasoning models work by generating intermediate steps before answering. Those steps are tokens, and tokens are billed.

This is a genuine capability improvement — the model checks itself, tries approaches, discards them. It's also why the same question can cost an order of magnitude more on a reasoning model than on a standard one, even at identical listed prices. You're not paying for the answer. You're paying for the thinking that produced it.

For a beginner, here's the mental model that matters: the price list tells you the cost per token, not the number of tokens your feature will need. A product decision like "use the reasoning model for accuracy" is also a decision to multiply your token count, and nothing in the pricing page warns you.

Mechanism 2: Agents Re-read Everything They've Ever Said

This is the big one, and it's the least understood. Stanford's Digital Economy Lab studied agentic coding tasks and found they consume roughly 1,000 times more tokens than code chat or code reasoning (Stanford Digital Economy Lab, May 2026). Apollo's research puts the range at 100x to 1,000x versus a standard chatbot request (Apollo, June 2026).

Why so extreme? Not because agents produce more output. Because of input token accumulation.

Picture the loop. An agent takes a step, then before the next step it re-reads the original prompt, plus its own previous response, plus the new one. Step three re-reads steps one and two. Step twenty re-reads nineteen steps of history. The context it must ingest grows with every action, and you pay for input tokens every single time.

Three findings from that study should change how you plan:

• Cost variance within the same task reached 30x across runs. Not different tasks — the same task, run again. Agent paths are stochastic, so your unit economics have a long tail.

• Agents can't predict their own spend. The researchers found models consistently underestimate their token consumption.

• Difficulty doesn't predict cost. Tasks that feel trivial to a human can be computationally expensive, which the authors connect to Moravec's Paradox.

That last point has a sharp commercial edge. As the Stanford team put it, "You can't really price the agent well unless you can figure out the cost, but now you only see the token costs after everything is done." If you're building a product with flat-rate pricing on top of agents, that sentence is your risk register.

Mechanism 3: For the First Time, the Price of a Token Is Rising

Here's the part that breaks the standard narrative. Token prices are up more than 60% since December 2025, according to J.P. Morgan Private Bank (July 2026). Not down. Up.

The cause is physical. Demand for inference outran the supply of chips, memory and power all at once:

Rationing showed up alongside the price moves. Anthropic limited its newest model to roughly forty organisations, and OpenAI's CFO Sarah Friar said the company is "making some very tough trades at the moment on things we're not pursuing because we don't have enough compute" (Tunguz, 2026).

Apollo's conclusion is worth reading twice: demand may force "materially higher pricing" for leading-edge models, or "usage rationing."

Our read: the 280x decline and the 60% increase aren't contradictory, because they measure different things. The 280x tracks the falling cost of yesterday's capability — old quality keeps getting cheaper, reliably. The 60% rise is the cost of today's frontier, where everyone actually wants to build. Cheap intelligence is abundant; the intelligence you want is scarce. Most 2026 budgets quietly assumed both curves pointed the same way.

What This Means If You're Building Something

Treat token consumption as an architectural property, the way you'd treat latency or memory. Six moves, roughly in order of payoff:

1. Instrument spend per feature before you scale it. Not per month — per feature, per user action. You cannot optimise a number you only see on an invoice.

2. Route by difficulty. Most requests don't need your most expensive model. Old capability is where the 280x savings actually live, and it's genuinely cheap now.

3. Cap agent loops. Set a hard ceiling on steps and tokens per task. Given 30x run-to-run variance, an uncapped agent is an uncapped invoice.

4. Attack input tokens, not output tokens. Since accumulation is the driver, summarise history instead of replaying it, and prune what goes back into context each turn.

5. Stress-test your pricing against the tail. If you charge a flat fee per task, model the 95th-percentile run, not the median.

6. Assume flat-to-rising unit prices for frontier models. Planning on another 10x discount is now a bet, not a baseline.

[INTERNAL-LINK: building cost-aware AI features → practical guide to instrumenting and capping LLM spend]

What Happens Next?

Genuinely uncertain, and the honest sources say so. Epoch AI's own caveat is that the fastest declines happened most recently, "so it's less clear that those will persist." Gartner expects provider costs to fall over 90% by 2030. Apollo expects shortage-driven price pressure in the near term.

Those aren't incompatible. Hardware and algorithmic efficiency keep pushing the floor down over years, while physical scarcity pushes prices up over quarters. Which force you feel depends on your time horizon and whether you need the frontier.

The safest planning assumption: unit prices for last year's capability keep falling, unit prices for this year's capability don't, and your consumption grows faster than either.

Frequently Asked Questions

What is a token, in plain terms?

A token is a chunk of text — roughly three-quarters of a word in English. Models are billed per million tokens, separately for input (what you send) and output (what it returns). Output typically costs 3 to 10 times more than input across major providers in 2026.

Why do agents cost so much more than chat?

Input token accumulation. Before each action, an agent re-reads the original prompt plus its own prior responses, so context grows with every step. Stanford's Digital Economy Lab measured agentic tasks consuming about 1,000x the tokens of code chat, driven mainly by this re-reading.

Is the AI price decline over?

For frontier models, it has at least paused. J.P. Morgan recorded token prices rising over 60% since December 2025 amid the compute shortage. Cheaper older-capability models continue to get cheaper, so the answer depends entirely on which tier you need.

How do I estimate an AI feature's cost before building it?

Run it fifty times and look at the distribution, not the average. Stanford found up to 30x cost variance across runs of the same agentic task, and that models underestimate their own spend — so a single test run or a model's own estimate will both mislead you.

The Number That Actually Matters

Stop quoting the price per million tokens. It's the least predictive number in your AI budget.

The number that matters is tokens consumed per unit of work you deliver — per resolved ticket, per shipped PR, per answered question. That figure is rising as agents replace chat, it varies by up to 30x for identical work, and it's now being multiplied by a unit price that stopped falling.

None of this argues against building with AI. Cheap capability really is abundant, and the 280x drop is one of the great deflationary runs in computing. It argues for building with a meter attached — because the era where you could ignore the meter and trust the trend line ended sometime last December.

Sources