If you pick an open-weight host on price per million tokens, you are choosing on the one number that barely varies. On October 17, 2026, Together, Fireworks, Groq and Amazon Bedrock all list gpt-oss-120b at $0.15 per million input tokens and $0.60 per million output, and Together, Fireworks, Baseten and Moonshot itself all charge $3.00 and $15.00 for Kimi K3. The tool call that comes back is what varies. When Moonshot audited third-party hosts serving its own model, schema accuracy ran from 100% down to 71.96%.
Use Fireworks AI as the default for tool-calling and agent workloads on open-weight models. It was the only third-party host to score 100% schema accuracy in both of Moonshot's published vendor tables, it does not store prompts for open models by default, and it charges the reference rate. Use Groq when latency on gpt-oss-class models is the requirement and you have run your own tool-call test. Use DeepInfra for cost-sensitive batch work, because it labels its precision and undercuts the reference price. Use Amazon Bedrock when your AWS commit and a written 99.9% SLA outweigh output length. Avoid Together AI for tool-heavy work until it publishes better numbers: it had the lowest schema accuracy of any host in this comparison in both Moonshot tables, it labels no precision, and it keeps your prompts unless an admin turns that off.
| Host | Kimi K3, per 1M tokens (in / out) | gpt-oss-120b, per 1M (in / out) | Precision shown on OpenRouter | Moonshot K2VV schema accuracy (K2 0905 / K2-thinking) | Prompts stored by default | Verdict |
|---|---|---|---|---|---|---|
| Fireworks AI | $3.00 / $15.00 | $0.15 / $0.60 | Not labelled | 100% / 100% | No | Default pick for agents |
| Groq | Not offered | $0.15 / $0.60 | Not labelled | 100% (trigger F1 69.52%, under the 80% bar) / not tested | No, but up to 30 days of abuse logs unless ZDR is on | Pick for latency, after your own test |
| DeepInfra | $2.85 / $14.25 | $0.037 / $0.17 | mxfp4 (K3), bf16 (gpt-oss) | 100% / 86.91% | No | Pick for batch and cost |
| Baseten | $3.00 / $15.00 | $0.10 / $0.50 | fp8 (K3), fp4 (gpt-oss) | 72.49% / not tested | No | Pick for dedicated deployments |
| Amazon Bedrock | $3.00 / $15.00 global, $3.30 / $16.50 US | $0.15 / $0.60 | Not labelled | Not tested | Model makers can't access them | Pick for an AWS commit |
| Together AI | $3.00 / $15.00 | $0.15 / $0.60 | Not labelled | 71.96% / 84.63% | Yes, until an admin enables ZDR | Loser for tool-heavy work |
Prices were checked on October 17, 2026 on each vendor's own pricing page, except DeepInfra's gpt-oss-120b rate and Bedrock's US gpt-oss rate, which come from OpenRouter's endpoint listing the same day. The K2VV figures were measured on November 15, 2025 on Kimi K2 models, so they describe each host's serving stack at that point and are not a Kimi K3 measurement. They are still the only public, like-for-like tool-call audit across these hosts.
Why the Same Open Model Behaves Differently on Each Host
The same weights behave differently on each host because every host runs its own serving stack, and tool calling is where those differences surface first. Moonshot built the K2 Vendor Verifier (K2VV) after users reported that Kimi K2 behaved differently depending on whose API they called. It sends about 4,000 requests to each provider and compares the answers with Moonshot's official API on two measures.
Trigger similarity is the F1 score on whether the model decides to call a tool when the official API does. Schema accuracy is the share of tool calls whose JSON arguments pass validation. A host can score well on one and badly on the other.
The November 15, 2025 results for kimi-k2-0905-preview show the spread. Moonshot set the pass bar at 80% trigger F1, because its own API scored as low as 82.71%, and 84% on average, across repeat runs. DeepInfra, Fireworks and NovitaAI all returned valid JSON on 100% of their tool calls. Baseten managed 72.49% and Together 71.96%. Groq's calls were 100% valid, but it triggered only 1,042 tool calls against 1,274 from the official API, for a trigger F1 of 69.52%, below the bar. Nebius scored a 50.60% trigger F1. On kimi-k2-thinking, Fireworks again hit 100% schema accuracy, while DeepInfra fell to 86.91%, Google Vertex to 85.76% and Together to 84.63%.
Most of the third-party rows were tested through OpenRouter, according to the "API Source" column, so they reflect what a buyer routing through OpenRouter received that day.
Moonshot has since replaced the method with the Kimi Vendor Verifier, released alongside Kimi K2.6. It adds a pre-check that the host actually enforces temperature and top_p, because Moonshot traced many reported anomalies to misuse of decoding parameters. It also adds an AIME2025 long-output stress test aimed at KV-cache bugs and quantisation loss. The full suite took about 15 hours on two 8-GPU H20 servers.
Moonshot is not the only party to see this. OpenRouter launched its Exacto endpoints on October 21, 2025 after its tool-call telemetry showed accuracy varying across providers more than standard benchmarks suggest, and Kimi K2 0905 was one of the five launch models. Baseten wrote on November 5, 2025 that the worst providers on public benchmarks had a tool-call success rate "of only 7%". Baseten sells inference and has an interest in that framing, but its account of the causes matches what vLLM had documented a week earlier.
What Causes the Gap, and Which Hosts Disclose It
Four things cause the gap: quantisation, KV-cache precision, chat-template and parser bugs, and truncated context or output limits. Of the six hosts compared here, only DeepInfra and Baseten carry a precision label on OpenRouter's listings, and none publishes the other three.
Quantisation is the reduction of model weights to fewer bits so they fit on less memory and run faster. Both reference models already ship in low precision. The Kimi K3 model card lists MXFP4 weights with MXFP8 activations, applied through quantisation-aware training, on a 2.8T-parameter model with 104B active. gpt-oss-120b was post-trained with MXFP4 MoE weights. A host serving the released format is serving the reference. A host that converts it to another format is serving a different build, which may be better or worse, and you can only find out by testing.
OpenRouter's Kimi K3 endpoint listing on October 17, 2026 shows Moonshot, DeepInfra and Modal at mxfp4, Baseten at fp8, and Together, Fireworks and Amazon Bedrock as "unknown". For gpt-oss-120b, DeepInfra is listed at bf16, Baseten and Nebius at fp4, Venice at int4, and Groq, Together and Bedrock as "unknown". That is the only precision disclosure a buyer can easily find, and it comes from a router, not from the hosts' own pricing pages. Together's pricing page labels one serverless model (a Qwen3 235B build) as FP8 and says nothing about the rest, and Fireworks' serverless pricing page labels none.
Detecting a silent swap is cheap if you have a reference. The DiFR paper (Karvonen et al., November 2025) showed that comparing generated tokens against a trusted reference with the same seed detects 4-bit quantisation with AUC above 0.999 within 300 output tokens.
KV-cache precision is the number format used for the attention cache that holds context during generation. Cutting it saves memory on long contexts, and the errors it introduces build up over long outputs. None of the six hosts publishes its KV-cache format on a pricing page, which is why Moonshot added the long-output AIME test. Cache handling choices move accuracy in ways that are hard to predict, as our look at KV-cache eviction found.
Template and parser bugs cause the failures that look like model stupidity. vLLM's debugging write-up on Kimi K2 (October 28, 2025) found three bugs: vLLM dropped the add_generation_prompt argument so prompts were cut off after the user turn, empty content strings were turned into a list that the Jinja template printed literally, and the tool-call ID parser crashed on IDs like search:2 and threw away valid calls. Fixing them took successful tool calls from 218 to 1,007 out of 4,000 requests. gpt-oss has its own trap: it must receive OpenAI's harmony response format or its output breaks, per its model card.
Truncation is the easiest to check and the one most often missed. On OpenRouter's K3 listing, Together, Fireworks, DeepInfra and Moonshot allow 943,718 completion tokens, Baseten allows 262,144, Alibaba 131,072 and Bedrock 128,000. On gpt-oss-120b, Groq caps completions at 65,536 tokens against 117,964 at most hosts. A reasoning model that hits the cap mid-thought returns a truncated answer you will log as a model error.
What the Price Difference Looks Like on One Agent Workload
On a defined agent workload, Kimi K3 costs within 5% at every host compared here that offers it outside region-pinned tiers, while measured throughput on gpt-oss-120b differs more than sixfold. The workload: 1,000 agent turns, each with 8,000 uncached input tokens and 500 output tokens, at list price on October 17, 2026.
On Kimi K3, that costs $31.50 at Moonshot, Together, Fireworks, Baseten and Bedrock's global cross-region tier. DeepInfra costs $29.93 at $2.85 / $14.25. Bedrock's US cross-region tier costs $34.65 at $3.30 / $16.50, and Fireworks' US-pinned endpoint costs $47.25 at $4.50 / $22.50, a 50% premium for keeping inference in the US. Cached input is $0.30 per million at Moonshot, Fireworks and Together, so an agent loop with a stable system prompt narrows these numbers further. Fireworks bills batch jobs at 50% of serverless pricing.
On gpt-oss-120b, the same 1,000 turns cost $1.50 at Together, Fireworks, Groq and Bedrock, $1.05 at Baseten, and $0.38 at DeepInfra's OpenRouter-listed bf16 rate. At these prices the model bill is rarely the line item that matters.
Speed differs far more. Artificial Analysis measured gpt-oss-120b output speed at a 10,000-token input as 473.5 tokens per second on Groq, 361 on Azure, 238 on Baseten, 232 on DeepInfra's Turbo endpoint (43 on its standard one), 136 on Google Vertex, 87 on Bedrock and 74 on Together. The page does not date those speed figures; we read it on October 17, 2026, and Fireworks was not on it. Groq's model list claims 500 tokens per second for the same model, which is close to the independent figure. Groq does not list Kimi K3 at all.
Price per Correct Tool Call
The number to compare is cost per valid tool call: price per turn divided by the share of turns that produce a call your code can execute. A failed call is worse than the arithmetic shows, because the retry resends the whole context and the agent loses a step.
The break-even is simple. DeepInfra's Kimi K3 rate is 5% below Fireworks'. If DeepInfra's schema accuracy on your workload is more than 5 points below Fireworks', the cheaper host is the more expensive one.
As an illustration of the mechanics only, apply the November 2025 K2-thinking accuracy figures to today's K3 prices for the workload above. Fireworks at 100% costs $31.50 per 1,000 valid calls. DeepInfra at 86.91% costs $34.43. Together at 84.63% costs $37.22. Those accuracy figures came from a different model a year ago, so do not treat the dollar amounts as a forecast. Run the test in the section below and put your own accuracy numbers into the formula.
The Contract Terms That Matter More Than the Rate Card
For an enterprise, data retention defaults, regional pricing, an uptime commitment and a written precision commitment carry more risk than a few cents per million tokens, and the hosts differ sharply on the first one.
Fireworks' data security page says it "does not log or store prompt or generation data for open models, without explicit user opt-in", lists SOC 2 Type II, says it is HIPAA compliant, and offers a US-only serverless option. Groq does not retain inference data by default, but may log inputs and outputs for up to 30 days to troubleshoot reliability or investigate abuse unless you enable Zero Data Retention (ZDR), and stores retained data in Google Cloud buckets in the US. DeepInfra keeps inputs in memory only and logs request ID, cost and sampling parameters, though its data page names no certification. Baseten's security documentation describes a zero-retention posture for synchronous inference, lists SOC 2 Type II and HIPAA, and notes that async inference stores inputs temporarily until processing completes.
Together's ZDR documentation is the outlier. ZDR is off by default, Together stores prompts and model responses and may use them for product improvements, and an admin has to set "Store prompts and model responses" to No in the organisation's Privacy settings. The switch is not retroactive and disables passthrough models. If your team signed up on a credit card, check that setting today.
Bedrock's data protection page says model providers have no access to Bedrock logs or to customer prompts and completions. Bedrock is also the only host here where we found a published SLA: service credits of 10%, 25% and 100% when monthly uptime falls below 99.9%, 99.0% and 95.0%. We found no public uptime figure for Fireworks, Groq, DeepInfra or Baseten serverless, and Together's inference FAQ mentions a "defined SLA" only for provisioned throughput, without terms.
None of the pricing pages or data documents above commits to a serving precision. Ask for it in the order form: the precision you will be served, a notice period before it changes, and the right to re-run your evaluation when it does.
Who Should Not Pick Each Host
Every host here is the wrong choice for someone, and the rate card does not tell you who.
Do not pick Fireworks if you need a published uptime commitment before you sign, or if US-only processing is mandatory and the 50% US endpoint premium breaks your budget. Do not pick Groq if you need Kimi K3, completions longer than 65,536 tokens on gpt-oss-120b, or a model that calls tools as often as the reference does; its 69.52% trigger F1 means its decisions on when to call a tool diverged from the official API's, and it made 18% fewer calls. Do not pick DeepInfra if your procurement team needs a SOC 2 report from a primary source before a pilot, or for agentic tool use without your own test, given its 86.91% on K2-thinking.
Do not pick Baseten serverless for Kimi K3 workloads that generate more than 262,144 output tokens, and treat its 72.49% K2 0905 score as a reason to test rather than a verdict, since Baseten says it self-quantises some models and used agentic-style data to quantise DeepSeek v3.1. Do not pick Bedrock for long Kimi K3 reasoning traces, which its 128,000-token completion cap will truncate, or for latency-sensitive gpt-oss work at 87 tokens per second. Do not pick Together for tool-heavy agents at all, and never for regulated data until an admin has switched ZDR on.
Google Vertex and Microsoft Foundry belong in the same evaluation if your cloud commit lives there. Vertex scored 85.76% schema accuracy on K2-thinking in K2VV, and Google's gpt-oss-120b endpoint on OpenRouter was listed at $0.09 / $0.36 on October 17, 2026. We could not confirm a first-party Foundry rate for either model that day, so get it from your Microsoft account team.
How to Run the Vendor Check Yourself
You can reproduce Moonshot's audit with its open tooling and your own traffic. Run it before you sign and again whenever a host changes its stack.
- Clone K2VV and run its tool-call suite against the model maker's official API and two candidate hosts. Use the official API's own run-to-run variance as your pass bar, the way Moonshot did, not an absolute number.
- Build 50 to 200 tool-call scenarios from your own traffic, the range Baseten recommends. Score trigger F1 and JSON schema validity against the reference for each host.
- Run a long-output test at the settings the model card used for its benchmarks (temperature 1.0 for Kimi K3) with
max_tokensnear the host's cap. Quantisation and KV-cache shortcuts show up first in long reasoning traces. - Confirm the host enforces
temperatureandtop_pby sending extreme values and checking that output changes. This is the Kimi Vendor Verifier's pre-check. - Divide each host's price per turn by its valid-call rate, and choose on that.
The harness you wrap around the model changes these numbers too, so test with the harness you will ship.
What to Do About It
This Week:
- Pull OpenRouter's endpoint listing for the model you run and note each host's precision label and completion cap.
- If anyone in your organisation uses Together, have an admin check the "Store prompts and model responses" setting.
This Month:
- Run K2VV plus 100 of your own tool-call scenarios against the reference API, Fireworks and your current host.
- Rebuild your cost model on price per valid tool call.
Before Renewal:
- Put the serving precision, a change-notice period and a retention default in the contract.
- Re-run the check on every stack change the host announces.
The Bottom Line
Hosted open-weight inference is repeating a lesson cloud buyers learned with virtual CPUs: two products with the same name and the same price can deliver different work, and only your own benchmark shows the difference. In 2026 the rate card has converged. Kimi K3 is $3.00 / $15.00 at four of the five hosts here that offer it and gpt-oss-120b is $0.15 / $0.60 at four of six hosts, so cost now comes down to how many of your calls succeed. Fireworks is the safest default today on the only public evidence, and Together carries the most risk on both accuracy and retention.
Run the tool-call test before you sign, and price the calls that parse.
Continue Reading
- FP8 vs NVFP4 vs INT4-AWQ: AWQ Outran NVFP4 on 3 of 4 Models
- Inference Cost per Million Tokens: Price the Task, Not the Rate
- vLLM vs TensorRT-LLM vs SGLang: Default to vLLM
- LLM Data Residency: Which Providers Actually Keep Data In Region
- 3,400 Tokens/s Was Batch 1. At 100K Context, It's Batch 12.
- DeepSeek Swapped the Model. Your Eval Didn't Notice.