Pydantic Logfire Documentation — AI Observability, Evals and Prompt Management

by Pydantic

IntermediateDocumentationFreemium~4-6 hours to work through Get Started plus the AI Engineering track; ongoing reference after that

Turn an agent run into one OpenTelemetry trace — model calls, tool calls and the slow database query that actually caused the timeout.

Start LearningAdded Sep 12, 2026 · Updated Sep 12, 2026

Overview

Logfire's documentation is organised as practical tracks rather than an API dump. Get Started covers sending a first trace, core concepts, a role-based "choose your path" split (AI engineer, backend and SRE, platform and security, product and growth, solo builder), and connecting a coding agent to Logfire's own MCP server. The Instrumentation section documents the Python SDK in depth — manual and auto tracing, metrics, distributed tracing, sampling strategies, and scrubbing sensitive data — alongside TypeScript (Node, browser, Cloudflare Workers, Next.js, Express, Deno, Vercel AI SDK) and SDKs for Rust, Go, .NET, Java, Ruby and PHP. Integrations covers roughly sixty libraries including FastAPI, Django, Flask, Starlette, Litestar, SQLAlchemy, psycopg, asyncpg, PyMongo, Redis, HTTPX, Celery, Airflow, structlog, loguru and pytest. The AI Engineering section is the reason an AI engineer reads it: per-provider instrumentation for OpenAI, Anthropic, Google Gen AI, LiteLLM and OpenRouter; a documented support matrix for more than twenty-five agent frameworks including Pydantic AI, LangGraph, CrewAI, Agno, AutoGen, smolagents, Claude Agent SDK, Google ADK, Strands Agents, Mastra, Vercel AI SDK and Semantic Kernel; LLM panels and a prompt playground; an Evaluations track covering datasets and experiments, evals in code, live evals and human review; prompt management with versioning, templates, testing and staged rollout; and an AI Gateway for multi-provider keys, failover and cost control. Later sections cover dashboards, alerts, an SQL workbench, feature flags, self-hosted deployment, and migration guides from Datadog, Grafana Cloud, Honeycomb, New Relic, Dynatrace, Splunk, Elastic and Braintrust. Because everything is OpenTelemetry underneath, the model call and the downstream service call that caused the failure sit in the same trace — the distinction the whole product is built around. The Python SDK is MIT-licensed and open source.

At a Glance

Topic: Frameworks
Level: Intermediate
Format: Documentation
Cost: Freemium
Duration: ~4-6 hours to work through Get Started plus the AI Engineering track; ongoing reference after that
Provider: Pydantic
Hands-on: Yes — code/exercises
Certificate: None

What You’ll Learn

✓Instrument a Python or TypeScript agent so one run becomes a single OpenTelemetry trace
✓Read LLM panels to separate a prompt or model failure from a downstream service failure
✓Query production spans in SQL to find the most expensive and least reliable agent paths
✓Build evaluation datasets out of real production failures and run live evals against them
✓Version, template and roll out prompts without redeploying the application
✓Route several model providers through the AI Gateway with failover and cost controls
✓Scrub sensitive data and apply sampling strategies before telemetry leaves your process
✓Wire tracing into 25+ agent frameworks using the documented support matrix

Highlights

•Built on OpenTelemetry, so a trace spans the whole request — model call, database query, HTTP call — instead of only the LLM layer
•Written by the Pydantic team, so Pydantic AI instrumentation, evals, prompt management and the gateway are first-party rather than bolted on
•The Python SDK is MIT-licensed open source (about 4,500 GitHub stars); the platform offers enterprise self-hosted deployment
•Documented support matrix for more than 25 agent frameworks across Python, TypeScript, Go, Rust and .NET
•Explicit migration guides from Datadog, Grafana Cloud, Honeycomb, New Relic, Splunk, Elastic and Braintrust
•A cookbook section with end-to-end recipes: instrument and evaluate an agent, debug a slow tool call, roll out a prompt safely, alert on LLM cost

Who It’s For

Best For

✓AI engineers debugging agent runs that fail intermittently in production
✓Teams already using Pydantic AI or Pydantic models who want native, low-friction tracing
✓Platform engineers standardising LLM telemetry on OpenTelemetry rather than a vendor SDK
✓Backend developers who want one tool for both application APM and LLM observability

Prerequisites

•A working Python or TypeScript service that already calls an LLM
•Basic familiarity with tracing concepts — spans, traces and attributes
•SQL, for the query workbench and dashboard sections

FAQ

What is Pydantic Logfire Documentation — AI Observability, Evals and Prompt Management?

The official documentation for Pydantic Logfire, an OpenTelemetry-native observability platform for LLM and agent systems, written by the team behind Pydantic and Pydantic AI. It is aimed at engineers already shipping an agent or RAG service who need to work out why a run failed. Work through it and you can instrument a Python or TypeScript app, read LLM panels for a single agent run, query production traces in SQL, version and roll out a prompt safely, and convert a production failure into a regression eval.

Is Pydantic Logfire Documentation — AI Observability, Evals and Prompt Management free?

Pydantic Logfire Documentation — AI Observability, Evals and Prompt Management offers free content, with paid options for certificates or premium features.

What level is Pydantic Logfire Documentation — AI Observability, Evals and Prompt Management for?

Pydantic Logfire Documentation — AI Observability, Evals and Prompt Management is aimed at a intermediate audience. Recommended background: A working Python or TypeScript service that already calls an LLM, Basic familiarity with tracing concepts — spans, traces and attributes, SQL, for the query workbench and dashboard sections.

How long does Pydantic Logfire Documentation — AI Observability, Evals and Prompt Management take?

Expect roughly ~4-6 hours to work through Get Started plus the AI Engineering track; ongoing reference after that. Most learners work through it at their own pace.

What will I learn from Pydantic Logfire Documentation — AI Observability, Evals and Prompt Management?

You'll learn: Instrument a Python or TypeScript agent so one run becomes a single OpenTelemetry trace; Read LLM panels to separate a prompt or model failure from a downstream service failure; Query production spans in SQL to find the most expensive and least reliable agent paths; Build evaluation datasets out of real production failures and run live evals against them; Version, template and roll out prompts without redeploying the application; Route several model providers through the AI Gateway with failover and cost controls; Scrub sensitive data and apply sampling strategies before telemetry leaves your process; Wire tracing into 25+ agent frameworks using the documented support matrix.

Topics

observabilityopentelemetryllm-tracingevalsprompt-management

Sources

This page was written from 4 sources, 1 on domains other than pydantic.dev.