Agent Router
by Agentic AI Foundation (Agent Router, a Series of LF Projects, LLC)
Open-source AI gateway, formerly Envoy AI Gateway, for LLM and MCP traffic on Envoy
Agent Router is an Apache 2.0 AI gateway, formerly Envoy AI Gateway, that gives platform teams one OpenAI-compatible endpoint for model providers, self-hosted inference and MCP servers. It is for organizations already running Kubernetes that want credentials, token limits and failover in one place instead of inside every application.
Agent Router is an open-source AI gateway that puts one OpenAI-compatible API in front of hosted model providers, self-hosted inference and Model Context Protocol (MCP) servers. It was called Envoy AI Gateway until September 2026, when the project left the CNCF Envoy sub-project structure, joined the Agentic AI Foundation and took its current name. Code, maintainers and the Apache 2.0 license carried over, and deployed resources keep their names: the aigateway.envoyproxy.io API group, CRDs, Helm charts, container images and the aigw CLI are unchanged. Bloomberg proposed the effort and Tetrate is a primary contributor. The first release came in February 2025, and v1.0 on June 23, 2026 added a stability commitment for the v1beta1 control-plane API. v1.1 followed on August 21, 2026 with token counting, per-request credentials and OpenTelemetry GenAI tracing. v1.2, on October 6, 2026, is the first release under the new name and hardens the MCP gateway for multi-user use, with breaking changes to MCP session encryption, OAuth issuer matching and cross-namespace Secrets. The gateway runs on Envoy and Envoy Gateway, lists 22 providers, and applies token limits per team, app or model, along with provider fallback and centrally held credentials. A two-tier pattern separates global authentication and routing from finer control over self-hosted models. Its site names Alan, Bloomberg, LY Corporation, Nutanix, Tencent Cloud and Tetrate as adopters. The open-source core has no admin console or SSO for administration, according to competitor TrueFoundry. Tetrate sells Agent Router Enterprise with managed operations, per-team cost attribution and runtime guardrails.
A platform engineering team that already operates Kubernetes and Envoy and has to give many internal teams governed access to models and MCP tools.
Applications authenticate to one gateway, so provider keys, token budgets and failover rules are managed centrally and a model switch is a model-name change.
At a Glance
- Category
- Infrastructure & Cloud
- Pricing
- Free, Contact for pricing
- Target Market
- CTOs, Platform Engineering Leaders, Heads of AI Infrastructure, Enterprise Developers
- Deployment
- Open-source, Self-hosted
- Founded
- 2025
Key Features
- ✓OpenAI-compatible unified API
One endpoint reaches 22 listed providers, so switching models means changing the model name in the request.
- ✓MCP gateway
Aggregates tool catalogs from many MCP servers and filters them by requester, which keeps each agent's tool list small.
- ✓Token-aware rate limits
Sets token limits per team, application or model so one workload cannot consume a shared provider budget.
- ✓Central credential management
Agents authenticate only to the router while provider keys and cloud credentials stay in one managed location.
- ✓Two-tier gateway pattern
A first tier handles authentication and global routing, and a second tier gives finer control over self-hosted models.
- ✓GenAI observability
Records models used, token counts, time to first token and fallbacks using OpenTelemetry GenAI conventions without instrumenting agents.
- ✓Standalone aigw CLI
The aigw run command serves a local OpenAI-compatible endpoint on port 1975 for testing before a Kubernetes rollout.
Capabilities
Use Cases
- •Internal LLM platform
A central platform team publishes one endpoint so product teams consume approved models under per-team token limits.
- •Provider failover
Requests fall back to another provider during an outage, so applications keep serving without carrying their own retry logic.
- •Self-hosted inference routing
A second-tier gateway with endpoint picker support sends traffic to self-managed vLLM clusters alongside hosted provider traffic.
- •MCP tool governance
Security teams use CEL rules over JWT claims so each agent identity sees and calls only its authorized tools.
Ideal For
Best For
- ✓Platform teams standardizing LLM access on Kubernetes with Gateway API and GitOps workflows
- ✓Enterprises mixing hosted providers with self-managed inference through Kubernetes InferencePool
- ✓Multi-team environments that need per-team, per-app or per-model token limits
- ✓Security teams that want agents to hold a gateway credential instead of provider API keys
- ✓Agent platforms that must filter large MCP tool catalogs by requester
Not Ideal For
- ✗Teams without Kubernetes and Envoy experience, because production use adds Gateway API, Envoy Gateway, a controller and a rate-limit service to operate
- ✗Buyers who need an admin console, SSO for administrators or built-in chargeback from the open-source project alone, since TrueFoundry's review reports those are absent
- ✗Organizations that require a vendor SLA and 24/7 support without buying a commercial distribution, since the project itself offers community support
- ✗Teams expecting first-party PII, prompt-injection or moderation guardrails, which are bring-your-own in the open-source core per TrueFoundry
Integrations
Deployment
Market Analysis
Pros
- ✓Apache 2.0 license with no gated data plane
- ✓Named production use at Bloomberg, with LY Corporation managing multi-tenant self-hosted LLM traffic on it
- ✓Regular release cadence in 2026, with six versions from January through October
- ✓CEL authorization over JWT claims for MCP tools, which competitor TrueFoundry describes as strong
Cons
- ✗Production deployment requires Kubernetes, Gateway API, Envoy Gateway, the controller and a rate-limit service, and Envoy Gateway upgrades become your dependency
- ✗No admin UI, so model onboarding, key rotation and quota changes are YAML manifests, per TrueFoundry's competitor review
- ✗Budgets are per gateway and cost data stops at metrics, so per-team chargeback must be built, per TrueFoundry
- ✗v1.2 contains breaking MCP changes, including random session encryption seeds that end active sessions on upgrade
- ✗No published latency benchmark and almost no Hacker News discussion to validate production claims independently
Pricing
Open source (Apache 2.0)
$0
- ✓Full gateway data plane and control plane
- ✓MCP gateway
- ✓Community support through GitHub and Discord
Tetrate Agent Router Enterprise
Contact for pricing
- ✓Managed operations
- ✓Per-team cost attribution
- ✓MCP tool governance
- ✓Runtime guardrails
The Apache 2.0 project costs nothing to license, so the real cost is the engineering time to run Kubernetes, Envoy Gateway and upgrades. Tetrate does not publish Agent Router Enterprise prices. Nothing in the open-source data plane is metered.
Security & Compliance
Connect
Sources
This page was written from 8 sources, 7 on domains other than theagentrouter.ai.
- 1.theagentrouter.ai — theagentrouter.aivendor
- 2.github.com — ai gateway
- 3.tetrate.io — envoy ai gateway reaches v1 0 establishing the open source s
- 4.tetrate.io — envoy ai gateway
- 5.itbrief.news — tetrate releases envoy ai gateway 1 0 for ai traffic
- 6.truefoundry.com — envoy ai gateway review
- 7.releasebot.io — agent router
- 8.hn.algolia.com — search
Stay Ahead of the Curve
Weekly enterprise AI insights for technology leaders. No spam, no vendor pitches—unsubscribe anytime.
SubscribeRelated Products
Featherless.ai
Serverless inference for tens of thousands of open-weight models through one OpenAI-compatible API
Merge Gateway
Managed LLM gateway with cost-based routing, failover and per-customer budgets behind a single API
Clockwork.io
GPU fault-tolerance and fabric observability software that keeps AI training jobs running through failures
Runpod
Per-second GPU cloud for training, fine-tuning and serverless inference, from RTX 4090s to B300s