Pipecat Documentation — Open-Source Framework for Voice and Multimodal Agents
by Daily
Learn real-time voice agents properly — turn detection, interruptions, telephony and latency, not just an STT-LLM-TTS demo.
Overview
The docs split into five properties. Getting Started has a five-minute quickstart, 'Build Your Next Bot' and a Pipecat 1.0 migration guide. Learning Pipecat is the real curriculum — eighteen chapters covering your first agent, session initialisation, pipeline and frame processing, transports, speech input and turn detection, speech-to-text, context management, LLM inference, function calling, text-to-speech, pipeline termination, multiple LLM agents, controlling the UI, agent handoff, job coordination, distributed agents and proxy agents. Fundamentals then covers the production problems: interruptions, user-input muting, idle-user detection, STT latency tuning, context summarisation, transcripts, recording, metrics, voicemail detection, IVR navigation, custom FrameProcessors and a multi-agent worker-bus architecture. A dedicated Evals section covers the eval lifecycle, writing scenarios, eval suites and third-party platforms. Telephony documents PSTN, SIP and WebSocket integrations for Twilio, Telnyx, Plivo and Exotel. Deployment covers running bots locally and in production with four hosting patterns — VM per session, warm pool, managed runtime and telephony — on Fly.io, Cerebrium or Modal. The Server API reference catalogues 58+ speech-to-text services, 40+ LLM services, 60+ text-to-speech services and seven realtime speech-to-speech APIs including OpenAI Realtime and Gemini Live, alongside video avatars, memory, VAD and OpenTelemetry. Client SDKs exist for JavaScript, React, React Native, iOS, Android and C++, and Pipecat Flows adds node-and-function conversation modelling with state management.
At a Glance
- Topic
- Agentic
- Level
- Intermediate
- Format
- Documentation
- Cost
- Free
- Duration
- 5-minute quickstart; ~8-10 hours for the full 18-chapter Learning Pipecat path
- Provider
- Daily
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Assemble frames, processors, pipelines and transports into a working voice agent
- ✓Handle turn detection, barge-in interruptions and idle users in real-time conversation
- ✓Wire STT, LLM and TTS services and swap vendors without rewriting the pipeline
- ✓Use realtime speech-to-speech APIs such as OpenAI Realtime and Gemini Live
- ✓Ship telephony agents over Twilio, Telnyx, Plivo, or Daily PSTN and SIP
- ✓Write eval scenarios and suites for conversations using the built-in Pipecat Evals
- ✓Model structured multi-step conversations as nodes and functions with Pipecat Flows
- ✓Choose between warm-pool, VM-per-session and managed hosting patterns for production
- ✓Instrument latency and OpenTelemetry metrics to find where response time is lost
Highlights
- •14.5k stars, 2.5k forks, BSD-2 licensed and maintained by Daily — the reference open-source voice-agent stack
- •Deliberately vendor-neutral: 58+ STT, 40+ LLM and 60+ TTS services behind one interface, so a provider change is a config change
- •Telephony is documented properly — PSTN, SIP and four carrier WebSocket integrations — which is where most voice-agent projects actually stall
- •Ships its own conversational eval framework with scenarios and suites, something most agent frameworks still lack entirely
- •Client SDKs for JavaScript, React, React Native, iOS, Android and C++ cover the browser and mobile half of the problem
- •Honest caveat: the framework is Python-only server-side, and the Pipecat Cloud sections are managed hosting you can skip if you self-host
Who It’s For
Best For
- ✓Engineers building voice agents, phone support bots or realtime multimodal assistants
- ✓Teams that need vendor portability across speech and LLM providers rather than one stack
- ✓Developers who have shipped a text agent and are hitting latency and interruption problems
- ✓Anyone comparing Pipecat against LiveKit Agents or a provider's realtime API
Prerequisites
- •Comfortable Python, including asyncio — the server framework is Python-only
- •Basic understanding of streaming audio, WebRTC or WebSockets concepts
- •API keys for at least one STT, LLM and TTS provider to run the quickstart
FAQ
What is Pipecat Documentation — Open-Source Framework for Voice and Multimodal Agents?
The documentation for Pipecat, the BSD-2 open-source Python framework for real-time voice and multimodal agents maintained by Daily. It teaches the pipeline model — frames, processors, transports — then the hard parts most tutorials skip: turn detection, interruptions, latency tuning, telephony, evals and production hosting, so you finish able to run a voice agent people can actually interrupt.
Is Pipecat Documentation — Open-Source Framework for Voice and Multimodal Agents free?
Pipecat Documentation — Open-Source Framework for Voice and Multimodal Agents is free to access.
What level is Pipecat Documentation — Open-Source Framework for Voice and Multimodal Agents for?
Pipecat Documentation — Open-Source Framework for Voice and Multimodal Agents is aimed at a intermediate audience. Recommended background: Comfortable Python, including asyncio — the server framework is Python-only, Basic understanding of streaming audio, WebRTC or WebSockets concepts, API keys for at least one STT, LLM and TTS provider to run the quickstart.
How long does Pipecat Documentation — Open-Source Framework for Voice and Multimodal Agents take?
Expect roughly 5-minute quickstart; ~8-10 hours for the full 18-chapter Learning Pipecat path. Most learners work through it at their own pace.
What will I learn from Pipecat Documentation — Open-Source Framework for Voice and Multimodal Agents?
You'll learn: Assemble frames, processors, pipelines and transports into a working voice agent; Handle turn detection, barge-in interruptions and idle users in real-time conversation; Wire STT, LLM and TTS services and swap vendors without rewriting the pipeline; Use realtime speech-to-speech APIs such as OpenAI Realtime and Gemini Live; Ship telephony agents over Twilio, Telnyx, Plivo, or Daily PSTN and SIP; Write eval scenarios and suites for conversations using the built-in Pipecat Evals; Model structured multi-step conversations as nodes and functions with Pipecat Flows; Choose between warm-pool, VM-per-session and managed hosting patterns for production; Instrument latency and OpenTelemetry metrics to find where response time is lost.
Topics
Sources
This page was written from 3 sources, 1 on domains other than docs.pipecat.ai.