NeMo Gym Documentation: Environments for Evaluating and Training LLM Agents
by NVIDIA
Build agent environments and verifiers, run repeatable benchmarks, and feed the same setup into RL training.
Overview
NeMo Gym is NVIDIA's open-source (Apache-2.0) library for evaluating and improving models and agents inside environments, where an environment combines a dataset, an agent harness, a verifier that scores the attempt, and per-task state. The documentation starts with concepts and a quickstart that runs a tool-calling agent on five multiple-choice samples through the gym CLI and writes rollouts and aggregate metrics. From there it splits by job. Prepare Tasks covers JSONL datasets and Harbor-format task folders. Build Verifiers covers equivalence matching, execution and state checks, LLM-as-judge, and multi-reward scoring. Evaluate covers benchmarks, harness comparison, Welch confidence intervals, rollout health checks, and resuming runs. The build tutorials move from single-step to multi-step, stateful, and real-world environments. Training tutorials pair environments with Unsloth for a 30-minute GRPO example on one GPU, VeRL for a one-hour DAPO run, and NeMo RL for a 3 to 5 hour GRPO series on multi-step tool calling. Infrastructure and orchestration pages cover sandbox providers such as Docker, E2B, Daytona, and ECS Fargate, plus Slurm submission and multi-node vLLM. Version 0.7.0 shipped in October 2026, and the project moved to monthly releases in July. The GitHub repository had 1.3k stars and 461 forks on November 11, 2026. Outside reviews are scarce: a December 2025 Hacker News post drew 2 points and no comments, so the tutorial timings are NVIDIA's own.
At a Glance
- Topic
- Agentic
- Level
- Advanced
- Format
- Documentation
- Cost
- Free
- Duration
- Self-paced reference docs; training tutorials are listed at 30 minutes (Unsloth), 1 hour (VeRL), and 3-5 hours (NeMo RL)
- Provider
- NVIDIA
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Run a first agent evaluation with the gym CLI and inspect rollout outputs
- ✓Define tasks as JSONL datasets or Harbor-format task folders with their own verifiers
- ✓Write verifiers using equivalence matching, execution checks, LLM-as-judge grading, and multi-reward scoring
- ✓Build single-step, multi-step, and stateful environments with tools and per-episode session state
- ✓Compare agent harnesses and benchmark runs using aggregate metrics and Welch confidence intervals
- ✓Train models on Gym environments with GRPO in Unsloth or NeMo RL and DAPO in VeRL
- ✓Capture token IDs from external harnesses such as Claude Code for RL training
- ✓Run sandboxed evaluations at cluster scale with Slurm submission and multi-node vLLM
Highlights
- •One set of docs runs from task format through verifier, evaluation, and RL training
- •Training tutorials state time estimates, from a 30-minute single-GPU Unsloth example to a 3 to 5 hour NeMo RL series
- •Integrates existing environment libraries, including Aviary, Harbor, OpenEnv, Reasoning Gym, and Verifiers
- •Includes rollout health checks, plus a reverify command that recomputes rewards without rerunning inference
- •Has walkthroughs for coding benchmarks including SWE-bench Pro and Terminal-Bench 2.1
Who It’s For
Best For
- ✓AI engineers building evals for tool-calling or coding agents
- ✓Teams post-training open-weight models with RL on custom environments
- ✓Researchers comparing agent harnesses on a fixed benchmark
Prerequisites
- •Python 3.13.14 on Linux, macOS, or Windows via WSL2
- •An LLM endpoint such as OpenAI, Azure OpenAI, or self-hosted vLLM
- •Working knowledge of agents, tool calling, and RL post-training methods such as GRPO
- •A GPU for the training tutorials, although the library itself does not need one
FAQ
What is NeMo Gym Documentation: Environments for Evaluating and Training LLM Agents?
NeMo Gym's documentation teaches AI engineers and post-training researchers how to evaluate and improve models and agents using environments. After working through it, you can define tasks and verifiers, compare agent harnesses on benchmarks, run sandboxed evaluations on a cluster, and train a model with RL on your environments.
Is NeMo Gym Documentation: Environments for Evaluating and Training LLM Agents free?
NeMo Gym Documentation: Environments for Evaluating and Training LLM Agents is free to access.
What level is NeMo Gym Documentation: Environments for Evaluating and Training LLM Agents for?
NeMo Gym Documentation: Environments for Evaluating and Training LLM Agents is aimed at a advanced audience. Recommended background: Python 3.13.14 on Linux, macOS, or Windows via WSL2, An LLM endpoint such as OpenAI, Azure OpenAI, or self-hosted vLLM, Working knowledge of agents, tool calling, and RL post-training methods such as GRPO, A GPU for the training tutorials, although the library itself does not need one.
How long does NeMo Gym Documentation: Environments for Evaluating and Training LLM Agents take?
Expect roughly Self-paced reference docs; training tutorials are listed at 30 minutes (Unsloth), 1 hour (VeRL), and 3-5 hours (NeMo RL). Most learners work through it at their own pace.
What will I learn from NeMo Gym Documentation: Environments for Evaluating and Training LLM Agents?
You'll learn: Run a first agent evaluation with the gym CLI and inspect rollout outputs; Define tasks as JSONL datasets or Harbor-format task folders with their own verifiers; Write verifiers using equivalence matching, execution checks, LLM-as-judge grading, and multi-reward scoring; Build single-step, multi-step, and stateful environments with tools and per-episode session state; Compare agent harnesses and benchmark runs using aggregate metrics and Welch confidence intervals; Train models on Gym environments with GRPO in Unsloth or NeMo RL and DAPO in VeRL; Capture token IDs from external harnesses such as Claude Code for RL training; Run sandboxed evaluations at cluster scale with Slurm submission and multi-node vLLM.
Topics
Sources
This page was written from 6 sources, 3 on domains other than docs.nvidia.com.