OGX Documentation — Open-Source Generative AI Application Server (formerly Llama Stack)
by OGX (Open GenAI Stack)
Self-host one OpenAI-compatible server that fronts 50+ inference, vector-store and tool providers — swap models without touching app code.
Overview
OGX (Open GenAI Stack) is the project formerly known as Llama Stack; the rename, explained on the project blog, was motivated by two persistent misreadings — 'Llama' implied it only served Llama models when it supports 23 inference providers including OpenAI, Anthropic, Bedrock, Azure, Gemini, Mistral and Ollama, and 'Stack' implied a client-side framework like LangChain or LlamaIndex when it is in fact an HTTP server with a pluggable provider architecture. The documentation, dated 29 July 2026 against stable release v1.2.2, is organised around the API surface: `/v1/chat/completions` for streaming inference and tool calling, `/v1/responses` for server-side agentic orchestration with MCP server integration, `/v1/embeddings`, `/v1/vector_stores` and `/v1/files` for managed retrieval, `/v1/batches` for offline processing, `/v1/models` for discovery, plus versioned Skills bundles that agents can invoke. Roughly fifty pluggable providers are documented — 23 inference backends, 15 vector stores (FAISS, ChromaDB, Qdrant, Milvus, PGVector, Weaviate and others) and 7 tool runtimes covering file search, web search, Wolfram Alpha and MCP. Because it is language-agnostic over HTTP, any OpenAI-compatible client or agent framework can point at it, and it additionally implements the Anthropic Messages and Google GenAI Interactions API shapes so the native Anthropic and Google SDKs work unchanged. The accompanying arXiv paper (2608.14580, June 2026) documents the architecture, a Kubernetes Operator for production deployment, 242 contributors and roughly 4,000 commits over two years.
At a Glance
- Topic
- Frameworks
- Level
- Intermediate
- Format
- Documentation
- Cost
- Free
- Duration
- ~4-6 hours to run the quickstart, wire providers and ship a Responses-API agent
- Provider
- OGX (Open GenAI Stack)
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Stand up a self-hosted OpenAI-compatible server fronting 23 different inference providers
- ✓Use the Responses API for server-side agentic orchestration with tool calling and MCP
- ✓Swap vector stores — FAISS, Chroma, Qdrant, Milvus, PGVector, Weaviate — without editing app code
- ✓Wire a retrieval pipeline by combining the files, vector stores and file-search tool APIs
- ✓Call one server from the OpenAI, Anthropic Messages and Google GenAI client SDKs
- ✓Package and invoke versioned Skills bundles that agents can call at runtime
- ✓Deploy to production with the Kubernetes Operator instead of hand-rolled infrastructure
Highlights
- •Renamed from Llama Stack in 2026 and genuinely model-agnostic — the blog explains exactly why the old name misled people
- •An HTTP server, not a library: your application language and backend choices stay independent of each other
- •Around 50 pluggable providers spanning inference, vector IO and tool runtimes
- •Backed by a published architecture paper (arXiv 2608.14580) with 242 contributors and 8.4k GitHub stars
- •Implements three frontier-lab API surfaces — OpenAI, Anthropic Messages and Google GenAI Interactions
Who It’s For
Best For
- ✓Platform teams standardising many AI applications on one self-hosted backend
- ✓Engineers who need to change model or vector database without rewriting application code
- ✓Regulated or air-gapped environments that must self-host inference and retrieval
- ✓Anyone previously building on Llama Stack who needs to understand the rename and migration
Prerequisites
- •Familiarity with the OpenAI chat completions and Responses API request shapes
- •Comfortable running a Python service, Docker container or Kubernetes deployment
- •Basic understanding of embeddings and vector search to use the retrieval APIs
FAQ
What is OGX Documentation — Open-Source Generative AI Application Server (formerly Llama Stack)?
Official documentation for OGX, the open-source AI application server renamed from Meta's Llama Stack in 2026. It is an HTTP server rather than a Python framework: it exposes OpenAI-compatible chat completions, embeddings, files, vector stores, batches and a Responses API with server-side agentic orchestration, backed by pluggable providers. Aimed at platform engineers who want application code to survive a change of model, vector database or cloud.
Is OGX Documentation — Open-Source Generative AI Application Server (formerly Llama Stack) free?
OGX Documentation — Open-Source Generative AI Application Server (formerly Llama Stack) is free to access.
What level is OGX Documentation — Open-Source Generative AI Application Server (formerly Llama Stack) for?
OGX Documentation — Open-Source Generative AI Application Server (formerly Llama Stack) is aimed at a intermediate audience. Recommended background: Familiarity with the OpenAI chat completions and Responses API request shapes, Comfortable running a Python service, Docker container or Kubernetes deployment, Basic understanding of embeddings and vector search to use the retrieval APIs.
How long does OGX Documentation — Open-Source Generative AI Application Server (formerly Llama Stack) take?
Expect roughly ~4-6 hours to run the quickstart, wire providers and ship a Responses-API agent. Most learners work through it at their own pace.
What will I learn from OGX Documentation — Open-Source Generative AI Application Server (formerly Llama Stack)?
You'll learn: Stand up a self-hosted OpenAI-compatible server fronting 23 different inference providers; Use the Responses API for server-side agentic orchestration with tool calling and MCP; Swap vector stores — FAISS, Chroma, Qdrant, Milvus, PGVector, Weaviate — without editing app code; Wire a retrieval pipeline by combining the files, vector stores and file-search tool APIs; Call one server from the OpenAI, Anthropic Messages and Google GenAI client SDKs; Package and invoke versioned Skills bundles that agents can call at runtime; Deploy to production with the Kubernetes Operator instead of hand-rolled infrastructure.
Topics
Sources
This page was written from 3 sources, 2 on domains other than ogx-ai.github.io.