NVIDIA NeMo Guardrails Documentation

NeMo Guardrails is an open-source Python package (Apache-licensed, 6.9k GitHub stars, latest release v0.23.0, Python 3.10-3.13) that intercepts an LLM application's inputs and outputs and applies configurable checks without changes to the model backend or the app itself. Its organizing idea is five rail types placed at different points in the interaction: input rails validate or rewrite the user message, dialog rails steer the conversation via Colang flows, retrieval rails filter RAG chunks before they reach the prompt, execution rails guard tool inputs and outputs, and output rails inspect what the model produced. The documentation is structured as Get Started (overview, installation, use with AI agents, tutorials, guardrail catalog), then Configuring Guardrails and Running Guardrailed Inference, then operational chapters on evaluating guardrails, logging, observability and deployment options, plus a LangChain integration guide, a security-guidelines page and a CLI reference. Rails are declared in YAML and Colang, a deliberately small Python-like language for dialogue flows; both Colang 1.0 (the default) and 2.0 are documented. Dialog routing works by embedding canonical forms and their example utterances into a vector space and matching new input by semantic similarity, so flow selection is a fast vector lookup rather than an extra LLM call. The catalog ships LLM self-check rails, NVIDIA content-safety and topic-control models, jailbreak and prompt-injection detection, PII redaction, and third-party integrations including ActiveFence and PolicyAI. A CLI provides chat, server and evaluate commands, and the API is async-first with sync wrappers.

Topic

Frameworks

Level

Intermediate

Format

Documentation

Cost

Free

Duration

~4-6 hours to work through Get Started plus the configuration and evaluation guides

Provider

NVIDIA

Hands-on

Yes — code/exercises

Certificate

None

• ✓Place input, dialog, retrieval, execution and output rails at the right interaction stage
• ✓Write Colang flows with canonical forms and example utterances for intent matching
• ✓Configure a guardrail bundle in YAML and load it from the Python SDK
• ✓Wire built-in content safety, topic control and jailbreak detection from the catalog
• ✓Redact PII and validate tool-call inputs and outputs in an agent loop
• ✓Filter retrieved RAG chunks before they ever reach the model prompt
• ✓Run guardrailed inference behind the bundled API server and Docker image
• ✓Evaluate whether a rail configuration actually blocks what you think it blocks
• ✓Add logging and observability so blocked interactions are auditable in production

• •One of very few frameworks that guards tool calls, not just prompts and completions
• •Dialog routing uses vector similarity rather than an extra LLM call, so it adds little latency
• •Ships a catalog of pre-built rails plus third-party integrations instead of leaving policy to you
• •Documents evaluation of the guardrails themselves — the step most guardrail projects skip
• •NVIDIA states plainly that the built-in rails may not suit a given production use case, which sets honest expectations
• •Same open-source package also ships as a NeMo microservice, so a prototype has a supported production path

This page was written from 4 sources, 2 on domains other than docs.nvidia.com.

1. 1.docs.nvidia.com — indexvendor
2. 2.docs.nvidia.com — homevendor
3. 3.github.com — Guardrails
4. 4.pinecone.io — nemo guardrails intro