NVIDIA NeMo Agent Toolkit Documentation
by NVIDIA
Find out which tool in your agent workflow is actually burning the tokens — across any framework.
Overview
The NeMo Agent Toolkit is NVIDIA's open-source library for connecting, instrumenting and optimizing teams of AI agents, and the documentation — currently version 1.8 — is organised as About, Get Started, Build Workflows, Run Workflows, Improve Workflows, Components, Extend and Reference. Its central design claim is framework-agnosticism: rather than replacing LangChain, LangGraph, LlamaIndex, CrewAI, AutoGen, Microsoft Semantic Kernel or Google ADK, it wraps agents and tools from any of them as reusable components inside a declarative workflow, so a team can profile and evaluate a heterogeneous stack with one toolchain. Build Workflows documents the built-in agent architectures — ReAct, Reasoning, ReWOO, Tool Calling, Router, and sequential and parallel executors — plus Model Context Protocol support in both client and server directions. Improve Workflows is where the toolkit is most distinctive: a profiler that instruments an entire workflow down to individual agents, tools and token counts to expose latency and cost bottlenecks; an evaluation harness for offline accuracy validation against datasets; and observability exporters that ship spans to LangSmith, Phoenix, Weights & Biases Weave, Langfuse or any OpenTelemetry-compatible backend. Documented extension points cover hyperparameter and prompt optimization and reinforcement-learning fine-tuning of workflow components. The project is Apache-2.0, installed as pip install nvidia-nat, and ships a UI for interacting with and debugging workflows. It is a smaller community than the LangChain-adjacent ecosystem at roughly 2.6k GitHub stars, but development is active with around 1,449 commits on the develop branch.
At a Glance
- Topic
- Agentic
- Level
- Intermediate
- Format
- Documentation
- Cost
- Free
- Duration
- ~3-5 hours to get started and profile a first workflow; ongoing reference after that
- Provider
- NVIDIA
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Wrap agents from LangChain, LlamaIndex or CrewAI as reusable toolkit components
- ✓Compose declarative multi-agent workflows using ReAct, ReWOO and Router agents
- ✓Profile a workflow down to per-tool latency and input/output token counts
- ✓Locate cost and latency bottlenecks before an agent workflow reaches production
- ✓Run offline evaluations of agentic workflows against a labelled dataset
- ✓Export spans to Phoenix, Langfuse, Weave or any OpenTelemetry collector
- ✓Expose and consume tools over MCP as both a client and a server
- ✓Debug a running workflow interactively through the bundled UI
Highlights
- •Profiling is the real differentiator — most agent tooling shows you traces, this attributes token cost and latency to individual tools and agents
- •Framework-agnostic by design, so it instruments the stack you already built instead of asking you to rewrite it
- •Ships exporters for LangSmith, Phoenix, Weave and Langfuse rather than forcing you onto its own backend
- •Apache-2.0 and vendor-neutral despite the NVIDIA branding — it is not restricted to NVIDIA NIM endpoints
- •Smaller community than the LangChain ecosystem (~2.6k GitHub stars), so expect to read source code for uncommon paths
Who It’s For
Best For
- ✓Platform teams running several agent frameworks who need one observability and evaluation story
- ✓Engineers whose agent works correctly but costs too much per run
- ✓Anyone preparing an agentic workflow for a production cost and latency review
Prerequisites
- •A working agent or tool built in a supported Python framework
- •Python packaging comfort (pip/uv) and willingness to write YAML workflow config
- •Familiarity with tracing and observability concepts such as spans and exporters
FAQ
What is NVIDIA NeMo Agent Toolkit Documentation?
The NVIDIA NeMo Agent Toolkit documentation covers an open-source, framework-agnostic library for building, profiling and optimizing multi-agent workflows. It is written for engineers who already have agents running in LangChain, LangGraph, LlamaIndex, CrewAI, Semantic Kernel or Google ADK and now need to know where the latency and token spend actually goes. After working through it you can wrap existing agents as toolkit components, profile a workflow down to individual tools and tokens, run offline evaluations, and export traces to your existing observability backend.
Is NVIDIA NeMo Agent Toolkit Documentation free?
NVIDIA NeMo Agent Toolkit Documentation is free to access.
What level is NVIDIA NeMo Agent Toolkit Documentation for?
NVIDIA NeMo Agent Toolkit Documentation is aimed at a intermediate audience. Recommended background: A working agent or tool built in a supported Python framework, Python packaging comfort (pip/uv) and willingness to write YAML workflow config, Familiarity with tracing and observability concepts such as spans and exporters.
How long does NVIDIA NeMo Agent Toolkit Documentation take?
Expect roughly ~3-5 hours to get started and profile a first workflow; ongoing reference after that. Most learners work through it at their own pace.
What will I learn from NVIDIA NeMo Agent Toolkit Documentation?
You'll learn: Wrap agents from LangChain, LlamaIndex or CrewAI as reusable toolkit components; Compose declarative multi-agent workflows using ReAct, ReWOO and Router agents; Profile a workflow down to per-tool latency and input/output token counts; Locate cost and latency bottlenecks before an agent workflow reaches production; Run offline evaluations of agentic workflows against a labelled dataset; Export spans to Phoenix, Langfuse, Weave or any OpenTelemetry collector; Expose and consume tools over MCP as both a client and a server; Debug a running workflow interactively through the bundled UI.
Topics
Sources
This page was written from 2 sources, 1 on domains other than docs.nvidia.com.