Kong AI Gateway
by Kong Inc.
One gateway for LLM, MCP and agent-to-agent traffic, built on the Kong data plane you already run
Kong AI Gateway extends Kong's API gateway into a control point for AI traffic, governing calls to LLM providers, MCP servers and agent-to-agent protocols from a single policy layer. It is aimed at platform and security teams who need PII sanitisation, token quotas, semantic caching and audit trails across every model an organisation's applications and agents call.
Kong AI Gateway is a set of AI-specific plugins layered on Kong Gateway, the Apache 2.0 Lua data plane that has 44,000 GitHub stars and is already deployed for API traffic in a large share of enterprises. It presents a provider-agnostic API so applications stay decoupled from OpenAI, Anthropic, Azure AI, AWS Bedrock, GCP Gemini, Databricks, Mistral, Hugging Face and other providers, and adds three distinct governance surfaces. For LLM traffic it ships AI Proxy and AI Proxy Advanced for routing and load balancing, an AI PII Sanitizer, AI Prompt Guard and AI Semantic Prompt Guard, semantic caching, AI RAG Injector, prompt templates and compressors, and guardrail integrations with Azure Content Safety, AWS Guardrails, GCP Model Armor and Lakera. For MCP it generates MCP tools and servers automatically from Kong-managed APIs and enforces authentication on MCP server access. For agent-to-agent traffic it captures A2A-specific telemetry — payloads, latency, token usage, errors — under centralised authentication and audit trails. Advanced token quota management sets limits per user, per model and per time window, and an observability dashboard tracks consumption, tool usage and token spend with predictive consumption models. Kong describes over 60 AI features in the community edition, so semantic security, caching and routing plus MCP governance are available in the open-source distribution rather than gated entirely behind Enterprise. On 11 August 2026 Kong published a working integration with NVIDIA NeMo Switchyard in which Switchyard acts as a per-request model-selection decision service while Kong keeps routing, credentials, policy and compliance; in Kong's own OpenThoughts-TBLite benchmark, moving the stage router confidence threshold from 0.3 to 0.5 cut frontier escalations from 85% to 17% for a 43.7% cost reduction per completed task. Kong Inc. was founded in 2009 as Mashape in Milan, is headquartered in San Francisco with 13 offices, passed $100M ARR in late 2023, and raised a $175M Series E at a $2B valuation.
The platform or API team that already runs Kong Gateway for its microservices and now needs the same policy, quota and audit controls applied to LLM, MCP and agent traffic without standing up a second, AI-only gateway.
One enforced policy point for every model call in the organisation — PII stripped, tokens quota'd per user and model, spend attributed, and every agent-to-agent hop logged — reusing the data plane and operational muscle memory you already have.
At a Glance
- Category
- Infrastructure & Cloud
- Pricing
- Freemium, Usage-based, Subscription
- Target Market
- CIOs, CTOs, Platform Engineers, Security Architects
- Deployment
- Hybrid, Self-hosted, Open-source, Multi-cloud
- Founded
- 2009
- Headquarters
- San Francisco, United States
- Team Size
- 500+
Key Features
- ✓Provider-agnostic AI Proxy
Presents one API to applications while routing behind it to OpenAI, Anthropic, Azure AI, AWS Bedrock, GCP Gemini, Databricks, Mistral or Hugging Face, so provider changes never reach application code.
- ✓AI PII Sanitizer and prompt guards
Strips personally identifiable information and blocks disallowed prompts before a request leaves your perimeter, plus semantic prompt and response guards for intent-level rather than keyword-level filtering.
- ✓Semantic caching and semantic routing
Serves semantically equivalent prompts from cache instead of re-billing the provider, and routes on meaning rather than exact match, which is where most of the direct token saving comes from.
- ✓Advanced token quota management
Sets and enforces token budgets scoped by user, by model and by time window, turning uncontrolled team-level model spend into an allocated, observable budget.
- ✓MCP traffic gateway
Auto-generates MCP tools and servers from Kong-managed APIs and enforces authentication on MCP server access, so agent tool use inherits the same authorisation as the underlying API.
- ✓Agent-to-agent (A2A) governance
Captures payloads, latency, token usage and errors for multi-agent traffic under centralised authentication and audit trails, which is otherwise an unobserved blind spot.
- ✓Guardrail provider integrations
Plugs into Azure Content Safety, AWS Guardrails, GCP Model Armor and Lakera Guard, so an existing safety vendor is enforced at the gateway rather than in each application.
- ✓NVIDIA NeMo Switchyard model routing
Delegates per-request model selection to Switchyard while Kong retains routing, credentials, policy and compliance, adding cost-aware routing without moving governance out of the gateway.
Capabilities
Use Cases
- •Enforcing one AI policy across many application teams
Central platform team sets PII rules, prompt guards and provider allowlists once at the gateway instead of reviewing every team's model integration.
- •Controlling runaway LLM spend
Per-user and per-model token quotas plus semantic caching cap what any single team can spend and remove repeat billing for equivalent prompts.
- •Governing MCP tool access for agents
Generate MCP servers from existing Kong-managed APIs so agents inherit the same authentication and authorisation the APIs already enforce.
- •Auditing multi-agent systems
Capture every agent-to-agent hop with payload, latency, token and error telemetry to produce the audit trail regulators and internal risk teams ask for.
- •Cost-aware model routing without losing control
Run NeMo Switchyard behind the gateway so cheap models absorb routine calls while Kong still holds credentials, quotas and compliance policy.
Ideal For
Best For
- ✓Centralising governance over LLM, MCP and agent-to-agent traffic when Kong Gateway is already the API control plane
- ✓Stripping PII and enforcing prompt guards before requests leave the perimeter for a third-party model provider
- ✓Token quota and cost attribution per user, per model and per time window across many internal teams
- ✓Decoupling applications from a specific model provider so switching vendors is a gateway config change
- ✓Adding intelligent model routing via the NVIDIA NeMo Switchyard integration while keeping credentials and policy in the gateway
Not Ideal For
- ✗AI-only teams with no existing API estate: TrueFoundry's pricing analysis argues you are buying a broad universal gateway — SOAP, XML, monolith support — of which AI workloads use a fraction
- ✗Python-native ML teams unwilling to take on Lua, which is the plugin language and, per the same analysis, means hiring a Lua specialist or spending engineering time on a niche language
- ✗High-volume agentic workloads on request-metered Konnect pricing, where a single user prompt can trigger 20+ internal API calls and each model endpoint counts as a separate billable service
- ✗Small organisations: AWS Marketplace reviewers repeatedly cite high licensing cost and enterprise features behind a paywall as prohibitive below a certain size
Integrations
Deployment
Market Analysis
Pros
- ✓Reuses an already-trusted data plane: Kong Gateway is Apache 2.0 with 44,000 GitHub stars, so the AI layer inherits proven operational tooling
- ✓Unusually broad plugin surface for AI governance — PII sanitiser, semantic and non-semantic prompt guards, RAG injector, prompt compressor and four third-party guardrail integrations
- ✓AWS Marketplace reviewers praise reduced API latency at millions of requests per day and deployment flexibility across on-premise, cloud and Kubernetes
- ✓Covers MCP and A2A traffic, which most competing AI gateways do not govern at all
- ✓Kong Inc. is a mature vendor — founded 2009, $100M+ ARR by late 2023, $2B valuation — rather than a one-round startup
Cons
- ✗Request-based Konnect metering is a poor fit for agentic traffic, where one prompt can fan out to 20+ internal calls and each model endpoint bills as its own service
- ✗Lua is the plugin language, which TrueFoundry's analysis argues forces Python-native AI teams to hire a Lua specialist or spend engineering time learning a niche language
- ✗You pay for a universal gateway — SOAP, XML, GraphQL, monolith support — that pure AI workloads largely do not use
- ✗AWS Marketplace reviewers cite unclear documentation lacking worked examples for advanced configurations, and high licensing cost that puts Enterprise features out of reach for smaller organisations
- ✗The data plane is over-provisioned for AI's low-RPS, high-latency token-streaming profile, so you provision for a traffic shape you do not have
- ✗Learning curve is real — reviewers say prior gateway experience is effectively a prerequisite and initial setup feels complex for beginners
Pricing
Kong Gateway (open source)
$0
- ✓Apache 2.0 licence, self-hosted only
- ✓60+ AI features including semantic security, caching and routing
- ✓Universal LLM API across OpenAI, Anthropic, Gemini, Bedrock, Azure AI, Databricks, Mistral, Hugging Face
- ✓MCP traffic governance and security
Kong Konnect (managed SaaS)
From ~$105/mo per gateway service
- ✓Approximately $34.25 per 1 million API requests
- ✓Approximately $720/mo base for dedicated network instances
- ✓Free tier limited by requests and services, suited to small proofs of concept
- ✓Managed control plane and analytics
Kong Enterprise (self-hosted)
Contact for pricing
- ✓Per-service licensing plus node-based capacity tiers
- ✓Contracts frequently exceed $50,000/yr for mid-sized deployments
- ✓Unlimited AI Gateway models, hybrid, dedicated and self-hosted gateways
- ✓Customisable analytics and dedicated support with custom SLAs
Kong publishes no unified AI Gateway price list. Konnect meters roughly $105/month per gateway service plus about $34.25 per million requests, with dedicated network instances around $720/month; self-hosted Enterprise is quoted per service with node capacity tiers and frequently exceeds $50,000/year for mid-sized deployments. The trap for AI teams is request metering: a single user prompt in an agentic workflow can trigger 20+ internal API calls, and each model endpoint counts as a separate billable service, so agent traffic consumes quota far faster than equivalent human API traffic. The open-source Kong Gateway carries 60+ AI features at no licence cost if you can self-host.
Security & Compliance
Connect
Sources
This page was written from 7 sources, 4 on domains other than konghq.com.
- 1.konghq.com — kong ai gatewayvendor
- 2.developer.konghq.com — ai gateway
- 3.github.com — kong
- 4.konghq.com — aboutvendor
- 5.konghq.com — llm routing kong ai gateway nvidia nemo switchyardvendor
- 6.truefoundry.com — kong gateway pricing architecture an analysis for ai teams 2
- 7.aws.amazon.com — prodview qwksr2wun4awo
Stay Ahead of the Curve
Weekly enterprise AI insights for technology leaders. No spam, no vendor pitches—unsubscribe anytime.
SubscribeRelated Products
NVIDIA NeMo Switchyard
Open-source model router that sends each agent step to the cheapest model that can still do the job
WEKA NeuralMesh
Microservices storage and memory fabric built for AI training and inference at exabyte scale
Tsuga
Bring-your-own-cloud observability that keeps telemetry, and its cost, inside your own AWS account
OpenObserve
Open-source, Rust-based observability on object storage — logs, metrics, traces and LLM telemetry in one binary