Merge Gateway
by Merge
Managed LLM gateway with cost-based routing, failover and per-customer budgets behind a single API
Merge Gateway is a managed LLM control plane for engineering teams running language models in production. It puts major providers behind one API endpoint and handles routing, failover, spend caps and data controls, which removes the need to build and operate that layer in-house.
Merge Gateway is a managed LLM gateway from Merge, the unified API company founded in 2020 by Shensi Ding and Gil Feig. It launched on March 31, 2026, according to TestingCatalog, as a product line separate from Merge's integration APIs. It sits between an application and its model providers: a team changes its base URL to gateway.merge.dev/v1, and existing OpenAI, Anthropic and LangChain clients keep working, with native Python and TypeScript SDKs also available. Merge lists OpenAI, Anthropic, Google, Grok, Mistral and Cohere as providers on its product page, and TestingCatalog also named AWS Bedrock at launch. Requests route by cost, latency or quality and shift to another model when a provider fails. A Build Your Own Router option lets teams weight routing with their own benchmarks and records an explanation for each decision. Budgets cap spend by project, team or customer tier in real time, and cost attribution consolidates provider invoices into one. Data controls include DLP, prompt injection protection and per-customer model and region policies. For companies that resell AI, an embedded mode gives each tenant its own routing policy, credentials, budget and usage reporting. Pricing is a 5% markup on provider costs, or $0.05 per million tokens for bring-your-own-key usage, with no seat charges. Gateway is managed only: there is no self-hosted option described. It competes with OpenRouter, LiteLLM, Kong AI Gateway and TrueFoundry. Merge last disclosed a $55M Series B, led by Accel, in October 2022.
A platform or AI engineering lead shipping LLM features to customers, who needs per-tenant budgets and failover and does not want to operate a gateway.
Model traffic, spend caps and provider failover move into one managed endpoint with a base URL change instead of a code rewrite.
At a Glance
- Category
- Infrastructure & Cloud
- Pricing
- Usage-based, Freemium
- Target Market
- CTOs, Platform Engineers, AI Engineering Teams, Enterprise Developers
- Deployment
- Cloud-only, API-based
- Founded
- 2020
Key Features
- ✓Routing and failover
Routes each request by cost, latency or quality and moves traffic to another model when a provider goes down, without code changes.
- ✓Build Your Own Router
Lets teams define the best model using their own weighted benchmarks or presets, and returns an explanation with every routing decision.
- ✓Real-time budgets
Caps spend by project, team or customer tier as requests arrive and compresses context to reduce token costs before limits are reached.
- ✓Cost attribution and invoice consolidation
Tracks spend by model, provider and team and rolls separate provider invoices into one, which simplifies chargeback for finance teams.
- ✓Data controls
Applies data loss prevention, prompt injection protection and per-customer model and region policies, with logs of routing decisions, costs and outcomes.
- ✓Embedded routing stack
Gives each end customer its own routing policy, credentials including bring-your-own keys, budget and usage reporting, all managed through APIs.
Capabilities
Use Cases
- •Provider outage protection
A production AI feature fails over to another model during a provider outage, which keeps the feature running without an engineering deploy.
- •Multi-tenant AI products
A SaaS vendor sets a separate budget and routing policy per customer and passes through billing with per-tenant usage reporting.
- •LLM cost reduction
A team routes routine requests to cheaper models and enforces spend caps; Windmill reports saving more than $10,000 a month this way, per Merge.
- •Migrating off direct provider keys
An engineering team centralizes API keys and audit trails by changing its base URL, so existing OpenAI and Anthropic clients keep working.
Ideal For
Best For
- ✓SaaS companies embedding AI that need per-customer routing policies, budgets and usage reporting
- ✓Engineering teams using several LLM providers that want failover without code changes
- ✓Finance and platform teams that need LLM spend attributed by model, provider and team on one invoice
- ✓Teams that want to route on their own benchmarks and keep an explanation for every routing decision
Not Ideal For
- ✗Organizations that must run the gateway in their own cloud, since Merge describes Gateway as managed and mentions no self-hosted option
- ✗Very high-volume teams with stable traffic, where a 5% fee on provider cost can exceed the cost of running open-source LiteLLM themselves
- ✗Teams using a single provider with no budget or tenancy requirements, who gain little from a routing layer
Integrations
Deployment
Market Analysis
Pros
- ✓Integration is a base URL change, and existing OpenAI, Anthropic and LangChain clients keep working
- ✓Spend controls and cost attribution are built in at project, team and customer level
- ✓Vendor lists SOC 2, HIPAA, ISO 27001 and GDPR, and the parent company has sold enterprise integration APIs since 2020
Cons
- ✗Managed only, so teams with strict in-cloud requirements cannot self-host it the way they can with LiteLLM
- ✗The 5% fee sits on top of provider costs, so it grows with volume, a tradeoff ToolJunction also notes
- ✗Launched in March 2026 with no Hacker News discussion found and no independent latency benchmark or user reviews to check vendor claims against
- ✗The customer logos and savings figure on the product page are vendor-supplied, and the page does not say which logos use Gateway as opposed to Merge's other products
Pricing
Free
$0
- ✓No platform fee, as listed by ToolJunction
Pro
Provider cost plus 5%
- ✓Routing and failover
- ✓Budgets and cost attribution
- ✓No per-seat charges
Bring your own key
$0.05 per million tokens
- ✓Use existing provider credentials
- ✓Available per tenant in the embedded stack
Merge charges a 5% markup on underlying provider costs with no per-seat fees, and bring-your-own-key usage costs $0.05 per million tokens. Both rates scale with volume, so model the fee against your monthly LLM spend. Merge's product page lists these rates, a free plan appears only in the ToolJunction listing, and limits on it were not published there.
Security & Compliance
Connect
Sources
This page was written from 6 sources, 3 on domains other than merge.dev.
Stay Ahead of the Curve
Weekly enterprise AI insights for technology leaders. No spam, no vendor pitches—unsubscribe anytime.
SubscribeRelated Products
Clockwork.io
GPU fault-tolerance and fabric observability software that keeps AI training jobs running through failures
Runpod
Per-second GPU cloud for training, fine-tuning and serverless inference, from RTX 4090s to B300s
LiveKit
Open-source real-time voice, video and AI agent infrastructure behind ChatGPT's voice mode, with a managed LiveKit Cloud
Akamai Inference
Managed open-weight model inference on Akamai Cloud with an OpenAI-compatible API, serverless or dedicated GPUs