Any model call nobody watches in real time should run on Batch, and anything a person waits minutes for should run on Flex, because both halve the token price at OpenAI, Google and AWS. Priority is worth buying for one thing: a customer-facing path where you already measure p99 latency and can show it costs you money. Anthropic is the outlier. It has Batch at 50% off and nothing in between, and Claude Sonnet 5.5 on Amazon Bedrock gets no discounted tier at all. All prices below were read from each vendor's own pricing page on November 16, 2026.
| Vendor | Batch | Flex (synchronous, sheddable) | Priority | Batch window | Verdict |
|---|---|---|---|---|---|
| OpenAI | 50% off | 50% off, same as Batch | 2x Standard ("Fast") | 24h, then expired |
Best Flex: no charge on a 429, documented fallback |
| Google Gemini API | 50% off | 50% off (preview) | +75-100% | 24h target, expires at 48h | Cheapest rates; Priority downgrades instead of failing |
| Amazon Bedrock | 50% off, select models | 50% off | +75% | 24h default, up to 168h | Good for Nova and open-weight models, useless for Claude Sonnet 5.5 |
| Anthropic | 50% off | None | Not sold for current models | 24h, most under 1h | Batch only; the loser for minutes-latency work |
What Batch, Flex and Priority Actually Mean
The three tiers sell the same model at different places in the vendor's queue. Batch is an asynchronous job: you upload a file of requests, the provider runs them when it has spare capacity, and you collect results later. Flex is a synchronous request that the provider may slow down or reject when standard traffic spikes. Priority (OpenAI now calls it Fast mode) jumps the queue for a premium.
The discount maths is close to identical across vendors. The differences that decide your bill are in the fine print: which models qualify, how long you wait, what happens to work that does not finish, and whether the discount survives prompt caching.
Batch, Flex and Priority Rates by Vendor (November 16, 2026)
On list price, Gemini 4 Hydrogen undercuts GPT-6 Sol and Claude Sonnet 5.5 on every tier, and every vendor's batch rate is exactly half its standard rate. Per 1M tokens, input / output:
| Model | Standard | Batch | Flex | Priority |
|---|---|---|---|---|
| OpenAI GPT-6 Sol | $2.00 / $10.00 | $1.00 / $5.00 | $1.00 / $5.00 | $4.00 / $20.00 |
| OpenAI GPT-6 Astra | $10.00 / $50.00 | $5.00 / $25.00 | $5.00 / $25.00 | $20.00 / $100.00 |
| Claude Sonnet 5.5 (Anthropic API) | $2.00 / $10.00 | $1.00 / $5.00 | n/a | n/a |
| Claude Opus 5.5 (Anthropic API) | $4.00 / $20.00 | $2.00 / $10.00 | n/a | Fast mode $8.00 / $40.00 |
| Gemini 4 Hydrogen | $1.50 / $7.50 | $0.75 / $3.75 | $0.75 / $3.75 | $2.70 / $13.50 |
| gpt-oss-120b on Bedrock (Sydney) | $0.1545 / $0.6180 | $0.0773 / $0.3090 | $0.0773 / $0.3090 | $0.2704 / $1.0815 |
Sources: OpenAI's pricing page lists Batch and Flex at the same rate for GPT-6 Sol, Astra and Luna, and notes that "Priority processing was renamed Fast mode on July 30, 2026." Anthropic's pricing page gives the Batch API "a 50 percent discount on both input and output tokens" and prices Opus 5.5 Fast mode at $8 / $40. Google's Gemini API pricing shows the Hydrogen rates. The Bedrock pricing page says "Flex tier pricing is at 50% discount to Standard tier pricing" and "Priority tier pricing is at 75% premium"; the only per-token tier prices it rendered for us were the gpt-oss models in Asia Pacific (Sydney), so treat that row as an illustration of the ratios rather than a US price.
Two pricing-page details move real money. Gemini 3.8 Flash costs $0.75 / $3.75 standard through December 31, 2026 and doubles to $1.50 / $7.50 on January 1, 2027, with batch and flex rates doubling with it. And every Gemini 4 Carbon rate doubles on December 1, 2026: standard input goes from $2.00 to $4.00 and output from $10.00 to $20.00, with batch rates doubling too. Any budget built on a December run rate is wrong in January.
Compare products on our own pages: Amazon Bedrock, Gemini, Google Vertex AI, Claude and GPT-6 Astra.
Completion Windows and What Happens When a Job Misses One
Every batch API promises a window, none of them guarantee it, and only the work that finished gets billed. Plan for expiry as a normal outcome.
At OpenAI, the Batch guide says each batch "completes within 24 hours (and often more quickly)" and the window "can only be set to 24h." Batches that miss it move to expired; unfinished requests land in the error file as batch_expired, and "you will be charged for tokens consumed from any completed requests." Output files expire after 30 days by default.
Anthropic's Message Batches docs say most batches finish in under an hour, and "batches expire if processing does not complete within 24 hours." Expired requests are not billed. Results stay downloadable for 29 days, and the same page notes processing "might be slowed down based on current demand," in which case "you might see more requests expiring."
Google's Gemini Batch API docs set a 24-hour target, and a job that runs or waits "for more than 48 hours" expires with no results to retrieve. That last clause matters: on Gemini an expired job can leave you with nothing, where OpenAI and Anthropic return the completed part.
On Amazon Bedrock, a batch job's timeoutDurationInHours defaults to 24 and accepts 24 to 168. Bedrock is the only one of the four that lets you set a longer window, which helps for a weekly backfill.
The windows do get missed. In September 2026 an OpenAI forum user reported almost all batches submitted September 3 to 5 expiring; OpenAI support confirmed the three sample batches "expired on September 5 after completing some of their requests." In March 2026, Gemini developers reported gemini-3.1-pro-preview batch jobs running past 35 hours and then expiring without an error, while older models finished in minutes.
Limits That Break a Migration
The limit that stops most migrations is model coverage, and the worst gap is Claude on Bedrock. The Bedrock model card for Claude Sonnet 5.5 marks Standard as supported and Priority, Flex, Reserved and Batch as not supported. A team that standardised on Claude through AWS for procurement reasons pays full price for every token, including its nightly evals. Amazon's own Nova 2 Lite supports Standard, Priority and Flex.
The rest of the list, by vendor:
- Anthropic has no Flex, and Priority is closed. The service tiers page says "Priority Tier capacity commitments are no longer available for purchase," and lists Opus 5.5, Sonnet 5.5 and Haiku 5.5 among the models Priority Tier does not support. Fast mode on Opus 5.5 buys output speed ("up to 2.5x higher output tokens per second") at 2x price, and it is "not available with the Batch API."
- Anthropic Batch is not eligible for zero data retention. The batch docs list ZDR as "not-eligible" with "29-day retention." If your contract depends on ZDR, Batch is off the table for that data.
- Request and file caps. OpenAI allows 50,000 requests and 200 MB per batch file; Anthropic allows 100,000 requests or 256 MB, whichever comes first; Gemini accepts input files up to 2 GB and inline batches under 20 MB.
- Tools. Bedrock batch jobs using the Converse format reject
toolConfigwith a 400 error; you have to switch to theInvokeModelformat to batch tool calls (Bedrock batch docs). Anthropic's batch API supports nearly every Messages feature exceptstream,speed,fallbacksandmax_tokens: 0. - Rate limits are shared. On Bedrock, "your on-demand quota for a model is shared across the priority, default, and flex service tiers" (service tiers), so a big Flex job eats headroom your interactive traffic needs. Gemini Flex also counts toward general rate limits, and Gemini Priority gets its own limit at 0.3x the standard one (Priority docs). Anthropic batch usage "does not affect rate limits in the Messages API," which is the cleanest separation of the four.
- Preview status. Gemini Flex and Priority are both labelled preview, and OpenAI calls Flex beta with "limited model availability."
How the Discounts Combine With Prompt Caching
At OpenAI and Anthropic the batch discount and the cache discount multiply; on Gemini it depends on the model. This is where a 50% saving becomes 70% or more on a workload with a long shared prefix, such as an eval rubric or an extraction schema.
- OpenAI lists a separate cached-input price per tier. GPT-6 Sol cached input is $0.20 standard and $0.10 on Batch and Flex. The Flex guide says tokens are "priced at Batch API rates, with additional discounts from prompt caching."
- Anthropic says its caching multipliers "stack with other pricing modifiers, including the Batch API discount." Sonnet 5.5 cache hits cost 0.05x base input, so $0.10 standard and $0.05 in a batch. The catch is that "cache hits are provided on a best-effort basis" in batches, with users seeing "30 percent to 98 percent" hit rates. Anthropic recommends the 1-hour cache for batches because the default 5-minute entry can expire before the batch reaches the request.
- Google halves the caching rate under Batch and Flex for Gemini 4 Hydrogen ($0.075 to $0.0375). For Gemini 3.1 Pro Preview the pricing page keeps caching at $0.20 on Batch and Flex, the same as Standard, and charges $4.50 per 1M tokens per hour of cache storage. Priority raises the cache-hit rate too: $0.135 on Hydrogen.
- Bedrock prices its tiers as a percentage of Standard. If you later buy Reserved capacity, AWS says your tokens-per-minute sizing must include
CacheWriteInputTokens, not only input tokens.
What the stacking is worth on an eval suite with 1B input tokens a month, 800M of them a shared prefix, and 100M output tokens:
| Model, Batch tier | Batch, no caching | Batch with prefix cached |
|---|---|---|
| GPT-6 Sol | $1,500 | $780 (all 800M hit at $0.10) |
| Claude Sonnet 5.5 | $1,500 | about $755 at a 98% hit rate, about $1,270 at 30% |
| Gemini 4 Hydrogen | $1,125 | $555 (all 800M hit at $0.0375) |
Plan around the Anthropic range. If your batch hit rate is near the bottom of it, caching saves you less than a third of what the spreadsheet promised. Measure cache_read_input_tokens on the first real batch before you put the savings in a forecast. We covered why cache hit rates collapse in our prompt caching teardown.
A Monthly Bill for Three Real Workloads
Moving an eval suite and a document pipeline off Standard halves their bill on every vendor that offers the tier; moving an interactive agent to Priority doubles it. Three defined workloads, priced at list rates without caching except where stated:
- Eval suite: 1B input and 100M output tokens a month, results needed by the next morning. Cheapest viable tier: Batch.
- Document pipeline: 3B input and 300M output tokens a month; a reviewer opens each result within 30 minutes of upload. Cheapest viable tier: Flex (Gemini targets 1 to 15 minutes per Flex request).
- Interactive agent: 500M input tokens (400M cache hits, 100M uncached) and 50M output tokens a month, user waiting in seconds. Cheapest viable tier: Standard. Priority shown for comparison.
| Workload | GPT-6 Sol | Claude Sonnet 5.5 (Anthropic API) | Gemini 4 Hydrogen | gpt-oss-120b on Bedrock (Sydney) |
|---|---|---|---|---|
| Eval suite, Standard | $3,000 | $3,000 | $2,250 | $216.30 |
| Eval suite, Batch | $1,500 | $1,500 | $1,125 | $108.20 |
| Doc pipeline, Standard | $9,000 | $9,000 | $6,750 | $648.90 |
| Doc pipeline, Flex | $4,500 | No Flex: Batch at $4,500 if the 24h tail is acceptable, else $9,000 | $3,375 | $324.60 |
| Agent, Standard | $780 | $790 | $555 | $108.15 (no caching) |
| Agent, Priority | $1,560 | Not available | $999 | $189.28 (no caching) |
The Sonnet agent figure treats the 100M uncached tokens as 5-minute cache writes at $2.50, which is what a cached agent loop pays on Anthropic; OpenAI and Gemini do not charge a write premium. The gpt-oss column is a smaller open-weight model and is not comparable on quality; it is there to show Bedrock's tier ratios.
The document pipeline is where Anthropic loses. On OpenAI and Gemini the same synchronous code, with one service_tier parameter, halves a $9,000 or $6,750 bill. On Anthropic the only discount needs a rewrite to an asynchronous job, and the result can take up to 24 hours, which breaks the 30-minute review promise. For a team on Claude, the realistic options are to pay Standard for that pipeline or to send it to a second provider. This is the same multi-provider argument we made for SLA coverage.
Flex Capacity Errors and the Retry Logic You Need
Flex is cheap because the vendor is allowed to refuse it, and both OpenAI and Google say so in their docs. OpenAI Flex can return "a 429 Resource Unavailable error code," and "you will not be charged when this occurs." Gemini Flex is "Best-effort (Sheddable)": requests "may be preempted or evicted" during standard-traffic spikes, return 503 at capacity or 429 on resource exhaustion, and get no server-side fallback to Standard.
Three things go wrong in a naive migration:
- Timeouts. Queued Flex requests are slow. OpenAI's SDK default timeout is 10 minutes and its examples raise it to 15; Google recommends client timeouts of "10 minutes or more." A gateway or load balancer with a 60-second idle timeout will kill these requests before the model answers.
- Double retries. OpenAI's SDKs already retry eligible rate-limit errors and honour
Retry-After. Wrapping them in your own retry loop multiplies attempts during exactly the capacity crunch you are retrying through. - No deadline. Exponential backoff with no cap turns a Flex job into an unbounded one. Every Flex call needs a deadline after which it falls back to Standard, which both OpenAI and Google's sample code describe.
A minimal pattern that works on both:
deadline = now() + FLEX_BUDGET # e.g. 20 minutes for a 30-minute review SLA
for attempt in range(MAX_FLEX_ATTEMPTS):
try:
return call(model, service_tier="flex", timeout=900)
except (RateLimit429, Unavailable503):
if now() + backoff(attempt) > deadline:
break
sleep(backoff(attempt) + jitter())
return call(model, service_tier="default", timeout=120) # pay Standard, keep the SLA
Then log which tier served each request, because that number is your real price. OpenAI and Bedrock return the served tier in the response, Bedrock also exposes ResolvedServiceTier in CloudWatch, and Gemini returns x-gemini-service-tier. If 40% of your "Flex" traffic falls back to Standard, your discount is 30%, not 50%.
Priority fails in a quieter way. OpenAI may downgrade Fast-mode requests when traffic ramps too quickly; they come back marked service_tier: "default" and are billed at Standard. OpenAI suggests raising traffic by no more than 50% every 15 minutes once you pass 1 million input tokens per minute, and says Fast mode for GPT-6 Astra "does not include a latency SLA." Gemini Priority also downgrades overflow to Standard rather than failing. Downgraded requests bill at Standard, so you pay the premium only when you get the speed, and only the served-tier log tells you how often that is.
On batch, the failure is the expired job. A vendor-written guide from Future AGI makes a fair practical point even though its quality-drift numbers come without data: failed-row recovery is your job, and the honest unit cost is batch spend plus sync retry spend divided by validated completed rows. Build the resubmit path before you need it.
Who Should Not Use Each Tier
Each tier hurts a specific buyer, and the vendor docs say which one.
- Skip Batch if an eval result decides whether a deploy goes out the same day, if your data needs zero retention on Anthropic, or if you run fewer than a few hundred requests a day and the job-state code costs more engineering time than it saves.
- Skip Flex if you have no retry deadline and fallback in code yet, if your Bedrock or Gemini quota is already tight (Flex shares it), or if you are on Claude, where it does not exist.
- Skip Priority if you cannot show what a slow response costs in revenue or SLA credits, if your traffic ramps in steps (OpenAI will downgrade the step), or if the workload is a batch or ETL job; OpenAI's own Fast mode guide says to avoid running those in Fast mode.
- Skip Claude on Bedrock for discount-dependent workloads. Sonnet 5.5 there is Standard only. Use the Anthropic API's Batch for the same model, or a Bedrock model with Flex.
The Decision: Criteria That Predict Regret
The question that predicts regret is how long a person waits for the output, and the answer maps almost mechanically to a tier.
- Nobody waits (evals, backfills, embeddings, nightly classification): Batch. On Anthropic this is your only discount, and it is a good one.
- Someone waits minutes (document review queues, report generation, back-office agents): Flex with a deadline and Standard fallback, on OpenAI or Gemini. On Claude, decide whether a 24-hour tail is acceptable; if not, pay Standard.
- Someone waits seconds: Standard with caching. Most of the agent bill above is output tokens, so trim output before you pay for speed.
- Seconds cost money you can count: Priority, on one path, behind a feature flag, with the served-tier log proving you got it.
What changes the answer: a vendor adding Flex to Claude or to Bedrock's Claude models (watch the model card's tier table), Gemini Flex leaving preview with a written latency target, or a January 1 price step like Gemini 3.8 Flash's that changes which model is cheapest before you change tiers. For capacity you would reserve rather than queue for, our provisioned throughput break-even covers the commitments.
This Week:
- Pull last month's token spend by API key and tag every workload "nobody waits", "minutes" or "seconds".
- Check the model card or pricing page for every model in the "nobody waits" bucket and confirm Batch exists for it on the platform you actually call.
This Month:
- Move one eval suite to Batch with a resubmit path for expired rows, and record cache hit rate on the first run.
- Put one minutes-latency pipeline on Flex with a deadline, Standard fallback and served-tier logging; read the fallback rate after two weeks.
Before the January 1, 2027 Price Steps:
- Re-price any Gemini 3.8 Flash workload at the doubled rates and decide whether Batch or Flex covers the increase.
The Bottom Line
Batch and Flex are the two largest discounts on an LLM bill that require no contract: 50% at OpenAI, Google and AWS, and 50% on Batch alone at Anthropic. Most teams leave them unused because the migration work is real: expiry handling, retry deadlines, served-tier logging. The same pattern played out with cloud spot instances, where the discount was public for years and the savings went to teams that built for interruption. Start with the eval suite, because it is the workload where nobody notices the wait.
Continue Reading
- Prompt Caching Savings: One Timestamp Turns 80% Into 6%
- Annual Azure and Google AI reservations break even at 84% and 75%
- OpenAI, Azure, Bedrock, Vertex and Anthropic SLAs Compared
- Inference Cost per Million Tokens: 2026 Provider Pricing Compared
- Gemini 4 Hydrogen Costs $1.87 a Task vs $7.77 for 3.8 Flash
- Bedrock vs Vertex AI vs Foundry: Your Cloud Commit Decides
- Best FinOps Tools for AI Spend: No Dashboard Fixes an Untagged Key
