Why 1,000x cheaper tokens tripled your AI bill

Here are the numbers nobody is putting together on one page. Frontier token costs dropped roughly 1,000x over 24 months. Enterprise AI spend went the other direction — the average enterprise AI budget climbed from $1.2M in 2024 to $7M in 2026, a 320% surge. Inference now accounts for 85% of AI budgets; training is no longer where the money is going. Microsoft’s Q2 FY26 capex landed at $37.5B with ~67% of that on short-life GPUs sized specifically for inference. This is not a training arms race anymore — it’s an inference gold rush, and your P&L is the gold.

The paradox resolves the moment you look at agentic workloads. A single customer-support ticket that cost 2,000 tokens in a chat-completion era now costs 30,000–60,000 tokens across an agent loop: retrieval, tool calls, planner/executor round-trips, verification, retries. Agentic tasks consume 5–30x more tokens per unit of work. Cheaper tokens removed the financial gatekeeper that was quietly capping your usage. The second you let agents plan and act, token volume decouples from revenue — usage scales with task complexity, not with customers. That’s why “per-token cost is dropping” is a trap line when you’re underwriting a 2027 budget.

The operators who have priced this in already are the ones showing real margin. ElevenLabs is at ~$330M ARR, ~$11B valuation. Retell AI cleared $50M ARR in under 18 months. Deepgram raised $130M at $1.3B. None of them are running naive single-model pipelines — they’re running tiered inference fabrics: distilled small models for 70–80% of routine traffic, caching on every RAG prefix, frontier models gated behind escalation routing. The voice AI unicorns have inference gross margin in the 60–75% band. The lookalike competitors who skipped this architecture are stuck at 25–40% and will not survive a funding environment that demands LTV:CAC proof by next quarter.