prompt caching

Priced on one normalised agent workload, a month of inference costs $370 on GPT-5.6 Luna and $10,725 on Claude Opus 5. Output, reasoning and cache-read rates explain the gap, not the input price.

Training a custom model costs about $30. Serving the same weights costs $326 to $28,908 a month depending on the platform. The fine-tuning-versus-RAG decision is a serving-contract decision, and the break-even is 683,000 queries.

A published trace of one 800-line pull request shows a coding agent billed roughly 156 million tokens to produce 289,000 — 98% of the volume was cache re-reads of context re-sent on every one of 512 turns. Coding-agent cost is set by turns per task and the width of each read, not by the model's rate card.

Anthropic cut Claude Fable 5.1 cache reads 75% to $0.25/MTok, half of Opus 5's rate on a model with twice the base input price. But Opus 5 stays cheaper until cache reads pass a third of your bill, and Anthropic's own savings ceiling of 45% falls short of the 50% crossover.

OpenAI cut GPT-5.6 Sol to $4/$20 per million tokens but guarantees the rate only "at least through November 21, 2026," with no successor published. The cut was asymmetric, so no two workloads saved the same amount — and the reversion is +50% on output, not +33%.

A production RAG pipeline at 10M tokens a day costs about $1,308 a month, 61% of it generation and one dollar of it embeddings. The line that wrecks the budget is document parsing, which swings from $2,100 to $90,000 over the same 2 million pages.

Self-host LiteLLM: per-team budgets and virtual keys are in the free open-source tier, while everyone else gates enforcement behind a sales call. Priced through one 50M-request workload, the platform layer ranges from $7 to $10,300 a month.

DeepSeek warned of a significant API price rise with no number and no date. Because the weights are MIT-licensed, the ceiling is already public: independent hosts serving the identical model charge 3-4x on posted rates and 40x on cache hits.

Manifest killed its four-tier LLM router after four months and 7,000 users, and the arithmetic explains why: cache reads bill at 10% of base input, so routing an agent step to a model 2.5x cheaper makes it 3.5x more expensive. Route at the session boundary, not the request.