LiveKit Agents Documentation
by LiveKit
Build realtime voice agents that hear, see and know when to stop talking.
Overview
LiveKit Agents inverts the usual agent-framework model: instead of a request-response loop over text, your Python or Node.js program joins a LiveKit WebRTC room as a full realtime participant, media streams through your AI pipeline, and results are published straight back into the room. The docs open with an introduction and key concepts — multimodality, logic and structure, the agent server, and model choices — then branch into a getting-started path built around a voice AI quickstart that runs in under ten minutes, a browser-based Agent Builder for no-code prototyping, a YouTube course series on production-ready implementations, and frontend integration guides. The building section covers agent sessions, tasks and task groups; workflows for structuring complex logic; LLM-callable tool definitions; pipeline nodes that process realtime audio and video; turn detection; agent handoffs; and external data integration for RAG-backed answers. Advanced material covers telephony over SIP, vision and multimodal input, MCP server integration, worker deployment including Kubernetes, and agent-server lifecycle management. Two things set it apart from generic agent frameworks. First, it ships its own transformer-based semantic turn-detection model that predicts when a speaker has actually finished, which is the single hardest part of voice UX and what most frameworks leave to a silence timer. Second, it has a native test framework with assertions on expected function calls plus an LLM judge, documented alongside the build guides rather than bolted on. Three run modes exist — console for terminal testing, dev with hot reload, and production. Plugins cover OpenAI, Deepgram, Cartesia, Google Gemini and avatar providers including Tavus, Bithuman and LemonSlice. The framework is Apache-2.0 (turn-detection weights under the LiveKit Model License) with 13.2k stars on livekit/agents, and the server is self-hostable.
At a Glance
- Topic
- Agentic
- Level
- Intermediate
- Format
- Documentation
- Cost
- Free
- Duration
- Voice AI quickstart in under 10 minutes; ~6-10 hours for the full build and deploy track
- Provider
- LiveKit
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Wire speech-to-text, an LLM and text-to-speech into a single realtime agent session
- ✓Use semantic turn detection so the agent stops interrupting the user mid-sentence
- ✓Define LLM-callable tools and pipeline nodes that process realtime audio and video
- ✓Hand off a live conversation between multiple specialized agents mid-call
- ✓Connect an agent to the public phone network over SIP telephony
- ✓Write automated tests that assert tool calls and grade replies with an LLM judge
- ✓Deploy agent workers in console, hot-reload dev and production modes, including on Kubernetes
- ✓Pull external data into a live conversation so the agent can answer from your own corpus
Highlights
- •One of the few agent frameworks built around realtime WebRTC media rather than a request-response transcript loop
- •Ships its own transformer-based semantic turn-detection model — the hardest problem in voice UX, solved for you
- •Native test framework with function-call assertions and an LLM judge is documented, not an afterthought
- •Apache-2.0 with a self-hostable server, so the docs are not a funnel into a single managed cloud
- •13.2k stars on livekit/agents with Python and Node.js SDKs maintained in parallel
- •SIP telephony is first-class, which most agent frameworks leave entirely to the reader
Who It’s For
Best For
- ✓Engineers building voice assistants, call-center agents or telehealth intake bots
- ✓Teams that need sub-second realtime media rather than a chat transcript loop
- ✓Developers adding phone or video channels to an existing LLM application
Prerequisites
- •Comfortable in Python or Node.js — the framework ships SDKs for both
- •Basic understanding of STT, LLM and TTS as separate model calls in a pipeline
- •API keys for at least one model provider such as OpenAI, Deepgram or Cartesia
FAQ
What is LiveKit Agents Documentation?
The official documentation for LiveKit Agents, an Apache-2.0 framework for realtime voice, video and physical AI agents in Python or Node.js. Your program joins a LiveKit room as a full participant, streams media through an STT-LLM-TTS pipeline, and speaks back. Aimed at engineers shipping voice assistants, call-center agents and phone bots, where latency and turn-taking matter far more than transcript quality.
Is LiveKit Agents Documentation free?
LiveKit Agents Documentation is free to access.
What level is LiveKit Agents Documentation for?
LiveKit Agents Documentation is aimed at a intermediate audience. Recommended background: Comfortable in Python or Node.js — the framework ships SDKs for both, Basic understanding of STT, LLM and TTS as separate model calls in a pipeline, API keys for at least one model provider such as OpenAI, Deepgram or Cartesia.
How long does LiveKit Agents Documentation take?
Expect roughly Voice AI quickstart in under 10 minutes; ~6-10 hours for the full build and deploy track. Most learners work through it at their own pace.
What will I learn from LiveKit Agents Documentation?
You'll learn: Wire speech-to-text, an LLM and text-to-speech into a single realtime agent session; Use semantic turn detection so the agent stops interrupting the user mid-sentence; Define LLM-callable tools and pipeline nodes that process realtime audio and video; Hand off a live conversation between multiple specialized agents mid-call; Connect an agent to the public phone network over SIP telephony; Write automated tests that assert tool calls and grade replies with an LLM judge; Deploy agent workers in console, hot-reload dev and production modes, including on Kubernetes; Pull external data into a live conversation so the agent can answer from your own corpus.
Topics
Sources
This page was written from 2 sources, 1 on domains other than docs.livekit.io.