W&B Weave Documentation

by Weights & Biases

IntermediateDocumentationFreemium~2-3 hours for the quickstart, tracing and evaluation tutorials; reference material thereafter

One decorator to trace an LLM app, and the same scorers for offline evals, live monitors and guardrails.

Start LearningAdded Aug 16, 2026 · Updated Aug 16, 2026

Overview

The docs are organised around two workloads. Tracing starts with a quickstart that adds @weave.op() to a function and captures every nested call into a hierarchical trace tree, then extends to generator functions, threads, sub-agents, batch logging, PII redaction, cost tracking, call querying and export, and a documented call schema; OpenTelemetry spans can also be sent in directly, so instrumentation is not locked to the SDK. Evaluation is the other half: weave.Model, weave.Dataset and @weave.op() scorers compose into weave.Evaluation, with built-in scorers covering exact match, regex, model-graded and embedding similarity plus MultiTaskBinaryClassificationF1, locally-run scorers, an evaluation logger for code-side results, comparison views, and static or dynamic leaderboards. Separate tracks cover annotation queues for human feedback, monitors and custom monitors on live traffic, guardrails that run scorers inline, a prompt Playground and Evaluation Playground, and self-managed deployment with ingest sampling. Integrations are unusually broad: agent harnesses (Google ADK, OpenAI Agents SDK, Claude Agent SDK, CrewAI, smolagents, PydanticAI, AutoGen, Agno, Haystack), frameworks (LangChain, LlamaIndex, DSPy, Instructor, Vercel AI SDK, verl, Verifiers, MCP) and roughly twenty model providers. SDKs are Python 3.10+ and TypeScript, documented in parallel; the client is Apache-2.0 on GitHub with active development, and a free W&B tier covers individual use. Cookbooks cover RAG evaluation, chain-of-density, OCR pipelines, audio and scorers-as-guardrails.

At a Glance

Topic: Frameworks
Level: Intermediate
Format: Documentation
Cost: Freemium
Duration: ~2-3 hours for the quickstart, tracing and evaluation tutorials; reference material thereafter
Provider: Weights & Biases
Hands-on: Yes — code/exercises
Certificate: None

What You’ll Learn

✓Instrument any function with a decorator and read the resulting nested trace tree
✓Send existing OpenTelemetry spans into Weave instead of rewriting instrumentation
✓Compose weave.Model, weave.Dataset and custom scorers into a reproducible Evaluation
✓Choose between built-in, LLM-judge and locally-run scorers for a given metric
✓Compare two prompt or model versions side by side and rank them on a leaderboard
✓Reuse the same scorers as production guardrails and monitors on live traffic
✓Trace sub-agents and threads so multi-agent runs stay legible in one view
✓Redact PII, track per-call token cost, and query or export calls programmatically

Highlights

•The same scorer object serves offline evaluation, online monitoring and inline guardrails — most tools make you write it three times
•First-class OpenTelemetry ingestion means you can adopt it without ripping out existing tracing
•Agent integrations go beyond frameworks to actual coding harnesses (Claude Agent SDK, Claude Code, Codex, Google ADK), which is rare in eval tooling
•Python and TypeScript SDKs are documented in parallel throughout rather than TypeScript as an afterthought
•Self-managed deployment, ingest sampling and bring-your-own-bucket media references are documented, so it is viable where traces cannot leave your VPC

Who It’s For

Best For

✓Engineers with an LLM prototype and no way to tell whether a change helped
✓Teams standardising evals across several agent frameworks at once
✓TypeScript LLM developers underserved by Python-only eval tooling
✓Platform teams needing self-hosted tracing for regulated data

Prerequisites

•Python 3.10+ or Node/TypeScript, plus a free Weights & Biases account
•A working LLM application or agent you already run end to end
•Basic grasp of evaluation concepts: datasets, ground truth, LLM-as-judge

FAQ

What is W&B Weave Documentation?

The official documentation for Weights & Biases' observability and evaluation layer for LLM applications and agents. It is aimed at engineers who have a working prototype and now need to see what it did, measure whether a change helped, and keep it honest in production. After working through it you can trace nested agent runs, build versioned datasets and scorers, compare model versions on a leaderboard, and run scorers as live guardrails.

Is W&B Weave Documentation free?

W&B Weave Documentation offers free content, with paid options for certificates or premium features.

What level is W&B Weave Documentation for?

W&B Weave Documentation is aimed at a intermediate audience. Recommended background: Python 3.10+ or Node/TypeScript, plus a free Weights & Biases account, A working LLM application or agent you already run end to end, Basic grasp of evaluation concepts: datasets, ground truth, LLM-as-judge.

How long does W&B Weave Documentation take?

Expect roughly ~2-3 hours for the quickstart, tracing and evaluation tutorials; reference material thereafter. Most learners work through it at their own pace.

What will I learn from W&B Weave Documentation?

You'll learn: Instrument any function with a decorator and read the resulting nested trace tree; Send existing OpenTelemetry spans into Weave instead of rewriting instrumentation; Compose weave.Model, weave.Dataset and custom scorers into a reproducible Evaluation; Choose between built-in, LLM-judge and locally-run scorers for a given metric; Compare two prompt or model versions side by side and rank them on a leaderboard; Reuse the same scorers as production guardrails and monitors on live traffic; Trace sub-agents and threads so multi-agent runs stay legible in one view; Redact PII, track per-call token cost, and query or export calls programmatically.

Topics

llm observabilityevaluationtracingguardrailsopentelemetry

Sources

This page was written from 3 sources, 1 on domains other than docs.wandb.ai.