Self-host LiteLLM, and buy something else only when you can name the specific feature it lacks. Your token bill is growing faster than your usage because nobody can attribute it, and the gateway you are shopping for is a chargeback system wearing a router's costume. At 50 million requests a month across three providers, the platform layer alone runs from $0 to more than $10,000 at published August 2026 list prices — and the expensive end is not doing more work.
That is the workload every number on this page is normalised to: 50 million gateway requests a month, three providers, twelve product teams that each need a defensible monthly figure, and roughly 60% of tokens sitting inside agent loops with long repeated prefixes. Vendors quote whichever metric flatters them — logs, models, seats, requests, a percentage of spend — so the only honest comparison is one workload priced through every meter.
| Gateway | Platform cost at 50M req/mo (list, Aug 2026) | Per-team budget enforcement | Self-host | Best for |
|---|---|---|---|---|
| LiteLLM | $0 open-source licence + your own infra; Enterprise quoted annually on capacity | Proxy, org, team, member, user and key budgets — in the free tier | Always | Almost everyone |
| Portkey | Published rate stops at 3M req ($310); ~$4,540 if extended to 50M | RBAC from $49/mo; enforcement via Enterprise | Yes, MIT core | Teams who refuse to run Postgres |
| Cloudflare AI Gateway | ~$7 — $5 Workers Paid plus ~$2 Logpush overage | Cost quotas per key with fallback, not chargeback | No | Cheap caching, analytics, edge |
| Vercel AI Gateway | $0 markup on tokens, ~$3,750 if you tag every request | Per-key reporting free; tags and quotas metered | No | Teams already deployed on Vercel |
| TrueFoundry | Enterprise quote; Pro Plus is $2,999/mo and SaaS-only | Yes, on Enterprise | Enterprise tier only | Air-gapped and regulated estates |
| Kong AI Gateway | ~$10,300 on Konnect Plus list rates | Yes, via an Enterprise-only plugin | Yes | Shops already standardised on Konnect |
| Helicone | $799/mo Team for SOC-2 and SSO, plus usage | No — observability, not enforcement | Yes, Apache 2.0 | Debugging traces, not budgets |
| OpenRouter | 5.5% of every credit purchase, or 5% BYOK past 1M req/mo | No | No | Prototyping. Not this job. |
What The Verdict Costs At 50 Million Requests
The spread between the cheapest and most expensive platform layer is roughly 1,500x, and it is driven entirely by which unit each vendor decided to meter. Run your own volume through each meter before you read another feature matrix.
LiteLLM publishes $0 for the open-source tier — "free forever, self-hosted," with virtual keys, users, teams, spend tracking, budgets, rate limits, fallbacks and Prometheus metrics all included, per its pricing page as of 8 August 2026. Enterprise is an annual contract priced on "annual gateway request capacity, deployment architecture, and support needs" and explicitly "never per token." There is no public number; you talk to sales, and there is a 30-day trial.
Portkey lists Production at $49 a month for 100,000 recorded logs, with "$9 overages per additional 100k requests". Read the ceiling on that rate before you extrapolate it: the comparison table scopes the overage to "$9/Month for Every 100K Up to 3M Requests," which tops out at $310 a month, and above 3 million Portkey publishes no rate at all. Extend the same meter to 50 million and it implies $4,540 a month for the privilege of recording what you already paid the provider for — but that is an extrapolation of ours, not a price Portkey quotes. Nobody at that volume pays list anyway; Portkey's own page puts Enterprise at "10 Mn Plus Recorded logs per month" with custom pricing. What the published rate does tell you is what the meter is measuring, and it is measuring your traffic.
Kong is worse on this workload for a structural reason. Konnect Plus includes 1 million API requests and charges "$200/month per additional 1 million", plus "$100/month per model" for up to 5 LLM models. Fifty million requests is 49 increments at $200 — $9,800 — before the model tax. Call it $10,300 a month at list, for a component whose job is to make a different bill smaller.
Cloudflare is the outlier in the other direction. Core AI Gateway features — "dashboard analytics, caching, and rate limiting" — are free, Workers Paid accounts get 10,000,000 logs per gateway, and Logpush is included to 10 million a month then "+$0.05/million." At 50 million requests that is the $5/month Workers Paid minimum plus about $2 of Logpush. Seven dollars.
Vercel AI Gateway charges no markup and no platform fee on tokens, including with BYOK — genuinely the cleanest token pricing here. The catch is narrower than it first looks. Custom Reporting, which is how you attach team tags and user IDs, costs "$0.075 / 1,000 tag/user ID/quota entity ID writes" and "$5 / 1,000 queries to the reporting endpoint" — so one tag on every one of 50 million requests is $3,750 a month, and three IDs per request is $11,250. But check whether you need tags at all before you budget for that. The reporting endpoint also groups by api_key_name, and an API key name is not a tag or user ID write. Twelve teams on twelve keys get a defensible monthly figure for the price of the queries alone — a dozen a month is six cents. Pay the tag meter only for attribution finer than the key: per-feature, per-environment or per-end-customer.
TrueFoundry publishes Developer at $0 for 50,000 requests, Pro at $499 for 1 million, and Pro Plus at $2,999 for 1 million requests and 25 users. Fifty million puts you in Enterprise, which is the only tier offering VPC, on-premises or air-gapped deployment. Helicone lists Pro at $79/month and Team at $799/month, with SOC-2 Type II, HIPAA and SAML SSO gated behind Team; request overage rates above the 10,000 free tier are not published on that page, which matters at this volume.
LiteLLM Wins Because Budgets Are Free
LiteLLM is the recommendation because the one capability you are actually buying — knowing which team spent what, and stopping them when they exceed it — is in the open-source tier rather than behind a sales call. That is not true of anything else on this list.
The budget hierarchy in the proxy docs runs global proxy, organisation, team, team member, internal user, virtual key, and end customer, with budget_duration accepting values like "30d" and a reset check every 10 minutes. A virtual key is a proxy-issued credential that maps a caller to a team and a spend ceiling without ever exposing the provider key — which is the actual fix for the shared OPENAI_API_KEY in the shared vault that nobody can attribute. Per-model budgets scoped to a key (model_max_budget) are marked Enterprise-only; the rest is free.
What Enterprise buys, per the pricing page, is SSO and SCIM, OIDC/JWT auth, audit logs, secret manager integration, key rotation, RBAC, org and team admins, a multi-region control plane, and support with SLAs up to 24/7. If your security team requires SCIM deprovisioning and an audit trail — most regulated ones do — you are buying Enterprise, and you should model that as an annual capacity contract, not a rounding error.
Do not pick LiteLLM if you have no platform team on call. You inherit a real operational surface. The production guide tells you to allocate 1 vCPU and 4Gi per worker as both request and limit, set proxy_batch_write_at: 60 so spend writes batch instead of firing per request, set disable_error_logs: True because "under sustained provider errors this bloats the spend logs table," cap database_connection_pool_limit: 10, and run Redis "as soon as you run more than one proxy instance." Past "roughly 1000+ requests per second, or 10+ instances" you must enable use_redis_transaction_buffer: true or spend tracking itself deadlocks and exhausts Postgres connections. That is a Tuesday-afternoon incident waiting for someone who has never operated it.
Portkey Is The Answer If You Won't Run Postgres
Portkey is the right buy for a team that wants virtual keys and fallbacks this week and has nobody to page when Postgres deadlocks — and its open-source core is genuinely good. The gateway repository is MIT-licensed, claims a 122kb footprint and "<1ms latency," routes to "1,600+ LLMs" with "50+ AI Guardrails," and runs from npx @portkey-ai/gateway, Docker, Cloudflare Workers or your own Kubernetes. You can self-host the data plane and still buy the control plane.
The Production tier at $49 is the best-value entry point in this comparison: 100,000 logs, 30-day log retention, unlimited prompt templates, LLM guardrails, semantic caching and role-based access control. For a team spending $20,000 a month on tokens with four squads, that is a rounding error against the visibility it returns.
Do not pick Portkey if your traffic is high-volume and low-value per request. Log-metered pricing tracks the exact variable you are trying to stop tracking upward. Yes, you can sample logging down to fit the tier — and then you have thrown away the attribution data you bought the product for. The meter and the mission are pointed in opposite directions, and that only stops mattering once you are on a negotiated Enterprise contract, which also gates SSO, VPC hosting and SOC-2 Type 2, GDPR and HIPAA.
Cloudflare Is Free And That Is The Catch
Cloudflare AI Gateway is the correct choice when you want caching, analytics and rate limiting at effectively zero cost and you do not need chargeback — and it is the wrong choice the moment finance asks for a per-team number.
The routing is more capable than the price suggests. Dynamic Routing supports if/else branching on request body, headers or metadata, percentage-based splits for A/B tests and gradual rollouts, rate-limit-triggered fallback, and cost-based routing that "enforces cost quotas (per your key, per period) and switches to fallback when exceeded." That last one is genuinely useful: it degrades a team to a cheaper model instead of failing them. It is not chargeback — it is a quota with a soft landing — and it is only reachable through the OpenAI-compatible endpoint, not the REST API.
Read the caching page before you assume the cache saves you anything. Cloudflare's cache is exact-match: "Any difference in the body — including messages, tools, or model parameters — will result in a separate cache entry." Minimum TTL 60 seconds, maximum one month, text and image responses only, semantic search explicitly a future plan, and the cache is volatile — "if two identical requests are sent simultaneously, the first request may not cache in time for the second." For agent traffic, where no two request bodies are ever byte-identical, the hit rate rounds to zero.
Do not pick Cloudflare if you need per-team budgets, self-hosting, or logs on 50 million requests in one gateway — the Workers Paid ceiling is 10,000,000 logs per gateway, so you are sharding gateways or sampling. And skip Unified Billing if you have negotiated provider commitments: "a 5% fee is applied to all credits purchased through Unified Billing," which on $200,000 of monthly token spend is $10,000 a month to avoid three invoices. Bring your own keys instead; provider rates through the gateway carry "no markup."
Kong And TrueFoundry Charge On The Wrong Meter
Kong and TrueFoundry are both credible products that lose this specific comparison on pricing shape, and both are defensible buys for a narrow buyer who already has the surrounding platform.
Kong's AI feature set is technically the deepest here. Its LLM cost-optimization cookbook stacks semantic routing to a cheap model for simple queries, semantic caching that returns a hit "with zero tokens consumed and sub-100ms latency," LLMLingua prompt compression that cut one demo prompt by "37%," and cost-based rate limiting that deducts real per-model input and output cost from a consumer's budget. Nothing else on this page compresses prompts. But the plugin that does token- and cost-based limiting is Enterprise-only — tier: ai_gateway_enterprise — so the cost control is behind the licence, and the licence meters per request and per model. Do not pick Kong unless you already run Konnect Enterprise and the AI gateway is an incremental plugin, not a new line item.
TrueFoundry is aimed squarely at the regulated estate: Enterprise supports SaaS, VPC, on-premises and air-gapped deployment with custom SLAs, and if you need an air-gapped gateway with a vendor on the other end of a contract, it belongs on your shortlist alongside its platform. Do not pick TrueFoundry if you need self-hosting under $3,000 a month — Pro Plus at $2,999 is SaaS-only by its own pricing page, so the deployment model you probably came for starts at Enterprise. Note also that TrueFoundry publishes a large library of comparison posts about its competitors; treat those the way you would treat any vendor writing about a rival, and price everything from the rival's own page.
OpenRouter Loses This Comparison
OpenRouter is the clear loser for enterprise multi-provider cost control, and the reason is contractual rather than technical. It charges 5.5% with a $0.80 minimum on credit purchases through Stripe, 5% via crypto, and for bring-your-own-key "the first 1M BYOK requests per-month are free, and for all subsequent usage there is a fee of 5% of what the same model and provider would normally cost on OpenRouter." On $200,000 of monthly token spend that is around $10,000 to $11,000 a month — more than Kong, for less control.
The deeper problem is that buying credits makes you OpenRouter's customer instead of Anthropic's, OpenAI's or Google's. You forfeit the committed-spend discount you were about to negotiate, the enterprise agreement, the data processing terms your privacy counsel already reviewed, and any BAA. Model pricing passes through "without any markup," which is true and beside the point: the markup is on the payment rail, and the thing you gave up is the contract. It is an excellent way to try eleven models in an afternoon. It is not a cost-control layer.
Two others do not lose so much as belong to a different category, and buying them for this job is the common mistake. Helicone is observability first — it will tell you exactly what happened and will not stop it happening, and SOC-2, HIPAA and SAML SSO sit at $799/month on Team. If you want traces and evals, that or Langfuse is the right shelf, and neither is a substitute for a budget ceiling. Vercel AI Gateway has the best token economics in this entire comparison — zero markup, zero BYOK fee — and reports spend per API key without a tag meter, which covers team-level chargeback outright. It is only attribution finer than the key that costs $0.075 per 1,000 tag writes, and that is where 50 million requests become $3,750.
Your Gateway Cache Is Not Where The Money Is
The single largest cost lever in this stack is the provider's own prompt cache, not the gateway's response cache — and aggressive cross-provider routing destroys it. This is the mistake that makes a gateway rollout increase the bill.
Provider-side caching is a discount on repeated prefixes. Anthropic prices cache reads at 0.1x base input tokens, 5-minute cache writes at 1.25x and 1-hour writes at 2x, with a default 5-minute lifetime refreshed free on each use. OpenAI applies its cached-input rate automatically with no code changes. On GPT-5.6 and later that covers prefixes of "at least 1,024 tokens" and "a cached prefix remains eligible for reuse for at least 30 minutes"; on earlier models the minimum "can range from 1,024 to 2,048 tokens."
Now read the fine print that decides your architecture. Anthropic's caches use "workspace-level isolation" and require "100% identical prompt segments," with a strict tools → system → messages invalidation chain — change a tool description and every cache dies. OpenAI states plainly that "prompt caches are not shared between organizations." Neither cache follows a request across providers, and no gateway can carry it across. So a router that sends turn one of an agent loop to Claude and turn two to GPT pays the 1.25x write premium twice and collects the 0.1x read exactly never — which is the trap we took apart in why prompt caching killed the multi-model router.
The rule that falls out: pin stateful agent loops to one model and route only stateless, short-prompt traffic. Classification, extraction, summarisation and embedding calls have no cached prefix to lose and are where cross-provider arbitrage genuinely pays, as the multi-provider routing case still holds. Configure your gateway so those are two different routing policies, not one. And keep an exit ready for the day list prices move — DeepSeek's price increase is the reminder that today's cheap tier is a business decision, not a physical constant.
The Latency Number Everyone Quotes Is Wrong
Gateway overhead is not the risk; proxy saturation under concurrency is, and the published benchmarks measure the first thing while production breaks on the second.
LiteLLM's own benchmarks report median overhead of 2ms, p95 of 8ms, p99 of 13ms and an average of 3.32ms at 1,170 RPS across four instances of 4 CPU and 8GB RAM, measured via the x-litellm-overhead-duration-ms header. Portkey claims "<1ms." Both are plausible and both describe processing time on an unsaturated box.
The number that should worry you is in a production performance report filed against LiteLLM v1.80.15: a reporter running 500 concurrent requests against vLLM on a 4vCPU/8GB VM measured throughput fall from about 16 requests per second direct to about 9 through the proxy, and disabling spend logging, tuning pgbouncer and adjusting ulimits changed nothing. That is one reporter on one configuration, not a general result — but it is the correct shape of the failure. The proxy is a CPU-bound Python hop in front of an I/O-bound call, and when it runs out of CPU it queues, and queueing does not show up in a median-overhead metric.
Size for concurrency, not for average RPS. Load-test at your p99 concurrency with spend logging on and the database attached, because that is the configuration that gives you the chargeback report, and it is the one the benchmark did not run.
Five Questions That Predict Regret
Every regret in this category traces back to one of five mismatches, and none of them are on a feature matrix. Answer these before you shortlist.
- Do you need enforcement or visibility? A dashboard that shows you overspent last month is a different product from a key that stops accepting requests at $5,000. If the answer is enforcement, Helicone and most observability tools are out regardless of how good they are.
- Does the vendor's meter match your traffic shape? High request count and low value per request punishes per-log and per-request pricing. Few large calls punishes nothing. Portkey at 50 million requests and Kong at 50 million requests are both list-price accidents of shape, not judgements of quality.
- Who holds the provider contract? If you want committed-spend discounts, enterprise terms, a BAA or zero-retention guarantees, the gateway must support BYOK and must not be reselling you tokens. This single question eliminates OpenRouter and Cloudflare Unified Billing.
- Is your traffic agentic or stateless? Sixty percent agentic means provider prompt caching dominates your bill and cross-provider routing is mostly a way to lose it. Mostly stateless means routing is your biggest lever.
- Do you have someone on call for this? Postgres deadlock remediation at 1,000 RPS is a real job. If nobody owns it, the $49 tier is cheaper than the free one.
What changes the answer: a negotiated committed-use discount large enough to make single-provider concentration cheaper than any routing scheme; provider caches gaining longer or portable TTLs, which would make routing safe on stateful traffic; or the gateway becoming your agent and MCP control point for tool allowlists and egress policy, at which point the decision moves from finance to security and the price stops being the deciding factor. That last shift is already visible in the move toward self-hosted coding gateways.
What To Do In The Next 30 Days
Start by measuring, because most teams shopping for a gateway do not yet know which of the five questions above they are answering.
This Week: Pull last month's provider invoices and try to attribute 80% of the spend to a named team. If you cannot — and most FinOps teams now managing AI spend cannot — you have confirmed the problem is attribution, not price. Then measure your cached-input token share from each provider's usage dashboard. Above roughly 40%, caching strategy outranks routing strategy and you should read the routing section again.
This Month: Stand up LiteLLM open source in a non-production namespace behind a single team's traffic. Issue virtual keys, set a budget_duration of "30d" with a real ceiling, and deliberately blow through it to confirm the key actually stops. A budget you have not tested is a dashboard. In parallel, run Portkey's $49 Production tier on the same traffic for a fortnight — it is cheaper than the meeting you will otherwise hold about it, and the comparison is worth more than any vendor's matrix.
Before Your Next Renewal: Price your real 12-month request volume through each vendor's published meter, not through their sales deck, and put the four numbers in one column. Take that column to your provider account teams before you sign anything — a credible multi-provider gateway in production is leverage in a committed-spend negotiation, and that discount is frequently larger than the entire gateway decision.
The Bottom Line
The LLM gateway market is repeating a pattern the API gateway market completed a decade ago: an open-source core that does the essential work, wrapped in commercial tiers selling SSO, audit logs and someone to call at 3am. Those tiers are worth real money to regulated enterprises and worth nothing to a team that just wants to know which squad burned $40,000 last month — and the pricing meters have not yet sorted themselves out, which is why the same 50 million requests can cost $7 or $10,300 depending on whose page you read.
Buy the smallest thing that enforces a budget. The gateway is not where your money goes; it is where you finally get to see it going. Start free, prove the ceiling actually stops a key, and let the invoice tell you when to upgrade.
Continue Reading
- Your AI Router Is Trading a 10x Discount for a 2.5x One
- Still Using One AI Provider? You're Overpaying by 67%
- Claude vs GPT vs Gemini: Stop Comparing Per-Token Prices
- AI Agents Doubled. Only 26% Know What They Cost.
- Agentic AI Costs 12x More Than CIOs Expected
- Your AI Vendor Just Became Your Biggest Risk
- Anthropic's Self-Hosted Gateway Rewrites the AI Coding War