A production RAG pipeline serving 10 million tokens a day costs about $1,308 a month in vendor spend. Generation is 61% of that. Embeddings — the line every cost model on the internet leads with — come to one dollar. And before you pay a cent of the $1,308, you will pay somewhere between $2,100 and $90,000 to turn your documents into text, on a choice most teams make in an afternoon and never price.
Here is the whole bill, at the workload defined in the next section, with every rate taken from a published vendor pricing page on 17 August 2026.
| Line item | What it is | Choice priced here | Monthly | Share |
|---|---|---|---|---|
| Generation | Answer tokens in and out | Claude Sonnet 5, $2/$10 per MTok | $792 | 61% |
| Guardrails | Content filter + grounding check | Bedrock Guardrails | $135 | 10% |
| Reranking | Top-50 candidates → top-12 | Cohere Rerank v3.5, $0.002/search | $120 | 9% |
| Non-prod indexes | Dev + staging copies | 2 × Pinecone Standard minimum | $100 | 8% |
| Vector store (prod) | 11.4 GB stored, 60k queries | Pinecone Standard | $50 | 4% |
| Observability | ~5 spans per query | Langfuse Core + overage | $45 | 3% |
| Retries | 5% of calls repeated | Same model | $40 | 3% |
| Evals | Weekly golden set + judge | Batch API | $25 | 2% |
| Embeddings | Corpus churn + query vectors | text-embedding-3-small | $1 | 0.1% |
| Recurring total | $1,308 | |||
| Ingest (one-time) | 2M pages parsed + embedded + upserted | Textract Detect Document Text | $2,158 | |
| Ingest, if you pick table extraction | Same 2M pages | Textract Analyze Document – Tables | $25,058 |
The verdict: spend your argument on the parser and the model, and stop shopping the vector database. Embeddings and production vector storage together are $51 of a $1,308 bill, and $50 of that $51 is a plan minimum rather than usage. Meanwhile the difference between two Amazon Textract API calls over the same 2 million pages is $22,900 — the equivalent of 29 months of generation at this volume.
What Workload These Numbers Assume
Every RAG cost estimate is worthless without the workload it was measured against, so here is the one used throughout. An internal knowledge assistant for roughly 3,000 employees:
- Corpus: 250,000 documents averaging 8 pages — 2 million pages, chunked into 2 million chunks of ~400 tokens, so ~800M tokens or ~3.2 GB of raw text.
- Embeddings: 1,024 dimensions, float32 → 8.2 GB of vectors; with chunk text carried as metadata, ~11.4 GB stored.
- Traffic: 2,000 queries a day = 60,000 a month.
- Per query: 600 tokens of system prompt, 3,600 tokens of retrieved context (12 chunks × 300), 200 of history, 200 of question = 4,600 input; 400 output. That is 5,000 tokens × 2,000 queries = 10.0M tokens/day, or 276M input and 24M output a month.
- Retrieval: top-50 candidates from the vector store, reranked down to the 12 chunks that go in the prompt.
- Churn: 5% of the corpus re-embedded each month.
Normalise your own numbers to this shape before comparing them to anything you read, including this. A vendor quoting "cost per query" without stating context length is quoting nothing.
Generation Is 61% of the Bill and the Only Lever That Matters
Model choice moves more money than every other decision in the pipeline combined. At 276M input and 24M output tokens a month, the spread across current models is nearly 24x:
| Model | Input / output per MTok | Monthly generation | vs Sonnet 5 |
|---|---|---|---|
| GPT-5.6-luna | $0.20 / $1.20 | $84 | −89% |
| gpt-5-mini | $0.25 / $2.00 | $117 | −85% |
| Claude Haiku 4.5 | $1 / $5 | $396 | −50% |
| Claude Sonnet 5 | $2 / $10 | $792 | — |
| GPT-5.6-terra | $2 / $12 | $840 | +6% |
| Claude Opus 5 | $5 / $25 | $1,980 | +150% |
Rates from the Claude pricing page and the OpenAI API pricing page, both checked 17 August 2026. Anthropic has confirmed that Claude Sonnet 5's $2/$10 introductory rate is now the standard price and the scheduled increase to $3/$15 on 1 September 2026 will not happen — which makes Sonnet 5 a third cheaper than Sonnet 4.6 per token. Read that saving carefully: the same page notes that Claude 4.7 and later models use a tokenizer producing roughly 30% more tokens for the same text, so on identical input the gap is nearer 13%. The same caveat runs through the table above, which holds the token count constant: Haiku 4.5 predates that tokenizer change and Sonnet 5 does not, so on identical source text Haiku's saving is nearer 62% than the −50% shown.
The important structural fact: RAG is an input-heavy workload. 92% of your tokens are input, because retrieved context dominates. That means output price barely matters and input price is almost the entire equation — the opposite of an agentic or coding workload, where long generations flip the ratio. When you evaluate a model for RAG, weight the input rate 11 to 1.
Which is why the honest recommendation is unglamorous: run the cheapest model that passes your eval, and re-run the eval every time a new tier ships. Moving this workload from Opus 5 to Haiku 4.5 saves $1,584 a month. If your golden set says the answers hold, that is the largest single cost decision available to you, and it takes an afternoon.
Embeddings Cost One Dollar a Month — Stop Modelling Them
Embedding is the line everyone builds a spreadsheet for and it is a rounding error. Here is the entire embedding bill.
The one-time index is 800M tokens. At $0.02 per million for text-embedding-3-small, that is $16 — and OpenAI's Batch API supports /v1/embeddings at a 50% discount with a 24-hour window, which is exactly the right shape for a corpus backfill, so $8. The premium option, text-embedding-3-large at $0.13 per million, is $104. Voyage's pricing runs $0.02/MTok for voyage-4-lite up to $0.12 for voyage-4-large, with the first 200M tokens free on every account — so the same backfill on voyage-4-large is $72 after the free tier.
Recurring is smaller still: 5% monthly churn is 40M tokens ($0.80), and 60,000 query embeddings at ~60 tokens each is 3.6M tokens ($0.07). Under a dollar.
The practical consequence is the opposite of what the arithmetic suggests. Because re-embedding is cheap, the cost of changing embedding models is not the embedding — it is the 2 million vector writes, the index rebuild, the shadow index you have to run while you compare retrieval quality, and the week of engineering time. Budget the migration, not the tokens.
Where to Put 2 Million Vectors, Priced the Same Way
Vector storage at this scale is close to a solved problem, and the price spread between options is a 385x range that has almost nothing to do with the vectors. Same corpus, same 60,000 queries:
| Option | What you actually pay for | Monthly | Don't pick it if |
|---|---|---|---|
| Amazon S3 Vectors | $0.06/GB-mo + $2.50/M queries + TB scanned | $2 | you need sub-100ms p99 retrieval |
| Pinecone (Standard) | $50/mo minimum, then $0.33/GB + read units | $50 (usage: ~$16) | you need three environments — each pays the minimum |
| Amazon Bedrock Knowledge Bases | $5.00/GB of raw data + $1.00 per 1,000 retrievals | $76 | you want to own chunking and the retrieval logic |
| Amazon OpenSearch Serverless (Classic) | 2 OCU minimum × $0.24/OCU-hour | $350 floor | you are under ~50M vectors |
pgvector on RDS db.r7g.2xlarge |
The instance, 24×7 | $698 (or $0 marginal) | the index is the only thing on the box |
Rates from the Pinecone pricing page, the Amazon Bedrock pricing page, the OpenSearch Service pricing page, the S3 pricing page and Vantage's RDS instance data, all 17 August 2026.
Four things worth pulling out of that table.
Pinecone's bill here is a minimum, not a price. Its documented cost model charges 1 read unit per GB of namespace with a floor of 0.25 RU per query; at an 11.4 GB namespace and 60,000 queries that is ~684,000 RU, which at $16–$18 per million — the page prints a range, because the rate varies by cloud and region — is $10.94 to $12.31, plus $3.76 of storage. Usage is $15–$16. You are billed $50 because Standard has a $50 monthly minimum. That is fine — it is also why the Enterprise tier's $500 minimum is 31x the workload it would be serving.
Amazon Bedrock Knowledge Bases prices the inverse of everyone else — $5.00 per GB of raw data per month plus $1.00 per 1,000 retrievals, with embeddings and the managed reranker included at no charge. A big corpus with light traffic is punished; a small corpus with heavy traffic is a bargain. It is the only option here where doubling your query volume doubles your storage-layer bill.
S3 Vectors is almost free and that is the whole catch. $0.68 of storage, $0.15 of query requests and $0.98 of data-processed charges. It is built for cheap, high-capacity, higher-latency vector search, not for a chat assistant that has to answer in under a second — but if you are building a nightly classification or enrichment job over the same corpus, running a second index there costs less than lunch.
Name the loser: OpenSearch Serverless Classic collections at this scale. A $350/month floor to serve a workload S3 Vectors handles for $1.81 and Pinecone for $50 is not a pricing decision, it is a default nobody costed. Its NextGen collections scale to zero and remove the floor entirely, which is the right answer if you are already committed to OpenSearch.
And the one most teams should actually take: pgvector on the Postgres you already run costs zero marginal dollars. The catch is memory, not money. An HNSW index runs roughly 2x the raw vector size — Crunchy Data measures 8 GB or larger for 1M rows of embeddings — so 2M vectors at 1,024 dimensions is ~16.4 GB of index that wants to stay resident. If your existing instance has 30 GB of headroom, this line is free. If it does not, you are buying a db.r7g.2xlarge at $0.956/hour and pgvector becomes the most expensive option on the list. The failure mode is loud and documented: pgvector users hit hnsw graph no longer fits into maintenance_work_mem mid-build and discover the sizing problem after the migration, not before.
Note also what you cannot put in the table. Qdrant Cloud's pricing page publishes no rate card — usage-based billing, a calculator, and "contact sales" for the Premium and Hybrid tiers. Cohere's pricing page defines a Rerank search unit as "one query with up to 100 documents to be ranked" but does not print the per-search price on the page — the $2.00 per 1,000 queries used in the table above is published on Amazon Bedrock's pricing page, by the reseller rather than the vendor. A vendor whose number you cannot read is a vendor you cannot compare, and in a procurement cycle that is a fact about the vendor, not about you.
The Parse Is the Budget: $2,100 or $25,000 for the Same Pages
This is the finding that should change what you do on Monday. Turning 2 million pages into text costs anywhere from nothing to $90,000 depending on which API call you make, and the difference is one parameter.
From the Amazon Textract pricing page, 17 August 2026, with its volume tiers applied to 2 million pages. Those tiers reset every calendar month, so the totals below assume the backfill runs inside one billing month — spread it over two and every page bills at the first-tier rate, making it $3,000 rather than $2,100 and $30,000 rather than $25,000:
| Parse method | Rate (first 1M / beyond) | 2M pages | Equivalent |
|---|---|---|---|
| Bedrock Knowledge Bases managed parser | included | $0 | — |
| Detect Document Text | $1.50 / $0.60 per 1,000 | $2,100 | 2.7 months of generation |
| Analyze Document – Tables (Layout free with it) | $15 / $10 per 1,000 | $25,000 | 31 months of generation |
| Analyze Document – Forms | $50 / $40 per 1,000 | $90,000 | 9 years of generation |
| Analyze Lending | $70 / $55 per 1,000 | $125,000 | — |
Twenty-five thousand dollars to parse a corpus is not a line item, it is a capital project — and it is being expensed by teams who spent three weeks choosing between vector databases that differ by $48 a month.
Two rules follow, and they are the highest-leverage things in this article.
Sample before you parse. Pull 500 random pages and count how many actually contain tables or key-value forms. If the answer is 6%, run Detect Document Text over all 2 million pages ($2,100) and Analyze Document – Tables over the 120,000 that need it ($1,800): $3,900 instead of $25,000. Table extraction on a corpus of prose is money set on fire for a feature the documents do not contain.
Then keep the parsed text forever. The extracted text is the single most expensive artifact in your pipeline and the one teams routinely throw away because it feels like an intermediate. Store it in object storage — 3.2 GB of text is cents a month — and every future re-chunk, re-embed and model migration replays from that instead of re-billing the parse. Teams who discard it pay the ingest cost again on every chunking experiment, which is how a $2,100 decision becomes a $10,000 one.
Prompt Caching Barely Helps a RAG Pipeline
Prompt caching is the first cost lever most teams reach for, and on RAG it does close to nothing — for a structural reason worth understanding before you build around it.
Caching is strictly prefix-based. Per the prompt caching documentation, the cache covers tools, system and messages in order, up to and including the block you mark, and a hit requires 100% identical content. In a RAG request the retrieved chunks change on every single query. They are 3,600 of your 4,600 input tokens — 78% of your input is uncacheable by construction.
Worse, the static part is usually too small to cache at all. Minimum cacheable prefix lengths are 512 tokens on Claude Opus 5, 1,024 on Claude Sonnet 5, and 4,096 on Claude Haiku 4.5 — and shorter prompts are silently processed without caching and no error is returned. A typical 600-token RAG system prompt is below the threshold on Sonnet 5 and nowhere near it on Haiku 4.5, so a team that "turned caching on" may be getting exactly zero benefit and have no signal that it failed.
Even in the good case the ceiling is low. A legitimate 1,200-token system prompt with few-shot examples is 26% of input, and a cache hit costs 0.1x base input, so the ceiling on the saving is ~23% of input cost — about $130 a month here. And that is the ceiling, not the expectation: the default cache lifetime is five minutes, and at 2,000 queries a day — roughly 1.4 a minute — the shared prefix stays warm through the working day but has to be rewritten at 1.25x base price after every off-peak gap. Worth taking, an order of magnitude less than picking a smaller model. If you are counting on caching, the agentic workloads where it genuinely dominates are a different shape entirely.
The lever that does work on the same axis: retrieve fewer, better chunks. Cutting from 12 chunks to 8 removes 1,200 input tokens per query — $144 a month off generation, plus $30 off guardrails, plus a measurable latency win. That is why the reranker earns its $120: its job is not accuracy alone, it is letting you send less.
The Six Lines Nobody Budgets
Every cost model has generation, embeddings and storage. Here is what gets left out, in order of how much it costs at this workload.
1. Guardrails, priced per character on context you already paid for — $135/month. Bedrock Guardrails charges $0.15 per 1,000 text units for content filters and $0.10 per 1,000 for contextual grounding checks, where a text unit is up to 1,000 characters and the grounding check meters the source, the query and the response combined. Your 3,600 tokens of retrieved context is ~14,400 characters, so a grounding check on every query is ~18 text units — $108 a month to re-read the same context you already bought from the model. In a regulated deployment this is not optional, and it means retrieving less context pays you twice.
2. Non-production copies of the index — $100/month. Dev and staging each need their own vector store, and on any plan with a minimum you pay it three times. Two extra Pinecone Standard tiers are $100. Two extra OpenSearch Serverless Classic collections would be $700 — more than production generation. This is the most reliably forgotten line in the entire stack.
3. Retries and failed generations — $40/month. Timeouts, rate limits and rejected structured outputs mean you pay full input price for answers nobody sees. 5% is a reasonable planning figure for a well-built pipeline; measure yours, because a badly-tuned JSON schema can push it past 15%.
4. Observability priced per span, not per query — $45/month. A single RAG query emits a trace plus retrieval, rerank, generation and scoring spans — roughly 5 billable units, not 1. Langfuse Cloud pricing includes 100k units on the $29 Core plan and charges $8 per additional 100k, so 300,000 units is $45. Self-hosting is free as software and roughly $100–200 of containers and Postgres. The trap is the multiplier: check what your vendor counts as a billable unit before you sign, because the difference between per-trace and per-span billing is 5x on identical traffic.
5. Eval runs and the LLM judge — $25/month. A 500-question golden set replayed weekly, plus a judge model scoring each answer, is ~15M tokens a month. Run it through the Batch API at 50% off — evals are the ideal batch workload and nobody does it.
6. The initial vector upsert — ~$50, once. Pinecone bills 1 write unit per KB with a 5 WU minimum per request; 2 million records at ~5.6 KB each is ~11.2M WU, or $45–$50 at Standard write-unit rates of $4–$4.50 per million. Trivial as a number, easy to be surprised by as an invoice.
And the line that is larger than all of the above put together: people. Half a platform engineer at a fully loaded $150,000 a year is $6,250 a month — nearly five times the entire vendor bill. Any cost conversation that optimises the $1,308 while adding a week of engineering work per month is arithmetic pointed the wrong way.
What Changes at 1M and 100M Tokens a Day
The shape of the bill changes more than the size of it. Same corpus, same architecture, three traffic levels:
| Daily tokens | Queries/month | Generation | Everything else | Total | Generation share |
|---|---|---|---|---|---|
| 1M | 6,000 | $79 | $235 | $314 | 25% |
| 10M | 60,000 | $792 | $516 | $1,308 | 61% |
| 100M | 600,000 | $7,920 | $3,441 | $11,361 | 70% |
Read it as three different problems.
Below ~5M tokens a day you are mostly paying minimums. At 1M/day, $204 of a $314 bill is plan floors and fixed costs — the vector store minimums, the observability plan, the eval suite. Optimising your model here is pointless; consolidating onto services without minimums is the whole game, and that is the one scale where S3 Vectors or pgvector-on-existing-Postgres is transformative rather than marginal.
At ~10M tokens a day the itemisation is worth doing, because no single line dominates and four different decisions each move 5–10% of the bill. This is the crossover, and it is why this workload was chosen.
Above ~50M tokens a day, model choice is the only conversation. At 100M/day generation is 70% and climbing; every other line is noise. At that point the levers are a smaller model, a distilled model, an open-weight model on committed capacity, and shorter context. Nothing else you do to the retrieval layer will show up on the invoice.
The corpus axis behaves differently again. Grow to 20M chunks and storage is still cheap on every option — but the pgvector HNSW index becomes ~164 GB, which no longer fits on a reasonable instance, and OpenSearch Serverless finally starts earning its OCUs. Query volume changes which model you buy; corpus size changes which store you buy. They are independent decisions and most teams conflate them.
Who Should Not Buy Each of These
The most useful column in any comparison is the one that tells you to walk away.
Don't run generation on Opus 5 unless your eval shows a measurable accuracy gap on your own corpus. At $1,980/month it costs 2.5x Sonnet 5 for a task — synthesising 12 retrieved chunks — that is largely extraction, not reasoning. Frontier reasoning is the wrong tool for a retrieval-grounded answer.
Don't run generation on GPT-5.6-luna or gpt-5-mini if your answers are compliance-facing or your grounding checks fail more than occasionally. The $700/month saving is real; so is the cost of one confidently wrong answer in a regulated context, and the smaller models are where citation fidelity degrades first.
Don't buy Pinecone if you need three environments and your production usage is under the minimum — you will pay $150 a month for $16 of work. Do buy it if you want the retrieval layer to be somebody else's on-call rotation, which is a legitimate thing to pay $50 for.
Don't buy Bedrock Knowledge Bases if you need control over chunking strategy, custom retrieval logic or a hybrid sparse-dense setup. Its free parser, free embeddings and free reranker are genuinely free and genuinely opinionated. Do buy it if your corpus is small, your traffic is heavy, and you would rather ship than tune.
Don't buy OpenSearch Serverless Classic collections below ~50M vectors. The 2-OCU floor is priced for a workload 50x this one.
Don't self-host pgvector if nobody on the team owns Postgres tuning. The index has to stay in RAM, maintenance_work_mem has to be sized before the build, and the true cost of self-hosting is operational, not licensing.
Don't buy Textract Analyze Document – Forms as a default. Sample your corpus first. It is 33x the price of text extraction and most enterprise document sets are prose.
Three Things to Do in the Next 30 Days
This week: Pull 500 random pages from your corpus and count how many contain real tables or forms. That single measurement decides whether your ingest costs $2,100 or $25,000, and it takes an afternoon. Then confirm your parsed text is being written to durable storage rather than discarded after chunking.
This month: Re-run your golden set against one tier below your current model — Haiku 4.5 or gpt-5-mini if you are on Sonnet 5 or Opus 5 — and put the accuracy delta and the dollar delta on the same page. Weight the input rate 11x the output rate when you compare, because that is your actual token mix. And check whether prompt caching is firing at all: if your system prompt is under the model's minimum cacheable length, it is silently doing nothing.
Before your next budget cycle: Itemise the six forgotten lines against your own invoices — guardrails, non-prod indexes, retries, observability spans, evals, initial upserts. Then put the loaded cost of the engineering time next to the total. If the vendor bill is $1,308 and the people cost is $6,250, you now know which number to spend the next quarter arguing about.
The Bottom Line
The RAG cost conversation has been running for three years on the wrong axis. Everybody models embeddings and vector storage because those are the parts that feel novel, and at 10 million tokens a day they are $51 of a $1,308 bill — $50 of which is a plan minimum. Meanwhile the parse, which feels like plumbing, swings by $22,900 on a parameter, and generation, which feels settled, swings by $1,896 on a model name.
This is the same mistake the industry made with data warehouses in 2015: obsessing over storage rates while compute quietly ate the budget. The bill has moved. The spreadsheet has not.
Price the parser. Price the model. Everything else is a rounding error with a sales team attached.
Continue Reading
- Pinecone vs Weaviate vs pgvector: Stay on Postgres
- Self-Host the Vector DB for Residency. Not for the Bill.
- Best RAG Platforms for Regulated Industries: Permissions First
- Your AI Router Is Trading a 10x Discount for a 2.5x One
- Best LLM Gateways for Cost Control: Self-Host First
- Datadog vs Arize vs LangSmith: Buy on the Billing Unit
- Claude vs GPT vs Gemini: Stop Comparing Per-Token Prices