garak — NVIDIA's LLM Vulnerability Scanner
by NVIDIA
Point it at a model and find out which jailbreaks, injections and leaks actually land.
Overview
garak — the Generative AI Red-teaming & Assessment Kit — is NVIDIA's open-source LLM vulnerability scanner, and the closest thing the field has to a Nessus for language models. You point it at a target and it runs a catalogue of probes, then applies detectors to the outputs to decide whether each attack landed. The plugin architecture has five parts: probes that generate the attacks, detectors that recognise failure modes, generators that adapt model backends, harnesses that orchestrate a run, and evaluators that score the results. Shipped probe families include dan for jailbreak personas, encoding for text-encoding injection, promptinject, gcg adversarial suffixes, glitch for glitch-token discovery, badchars for unicode and homoglyph perturbation, leakreplay for training-data leakage, malwaregen, packagehallucination, misleading claims, xss, and atkgen, which drives an attack model to generate probes adaptively rather than replaying a fixed list. Targets can be Hugging Face Hub models, OpenAI, AWS Bedrock, Cohere, Groq, Replicate, NIM, LiteLLM, GGUF models via llama.cpp, or any REST endpoint, so the same suite runs against a hosted API and a local build. Every run writes garak.log, a timestamped JSONL report and a hit log of confirmed failures. It is Apache-2.0 with roughly 8,900 GitHub stars, maintained by NVIDIA's AI Red Team, with releases landing roughly monthly — most recently v0.16.0, adding technique and intent annotation, context-aware scanning, a native Anthropic generator and an adaptive-attacks probe.
At a Glance
- Topic
- Frameworks
- Level
- Intermediate
- Format
- Documentation
- Cost
- Free
- Duration
- ~1 hour to your first scan; a full default probe sweep runs for hours
- Provider
- NVIDIA
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Run a first scan against an OpenAI, Hugging Face or REST-hosted target
- ✓Select probe families deliberately instead of always running the full default sweep
- ✓Understand how probes, detectors, generators, harnesses and evaluators fit together
- ✓Test for prompt injection, jailbreaks, encoding attacks and glitch tokens
- ✓Detect training-data leakage and package-hallucination risk in generated code
- ✓Read the JSONL report and hit log to separate real failures from noise
- ✓Write a custom probe or detector for a threat specific to your own application
- ✓Scan self-hosted GGUF and NIM deployments with the same suite as hosted APIs
Highlights
- •Maintained by NVIDIA's AI Red Team, with a published paper documenting the methodology
- •A large catalogue of probe families out of the box — you are not authoring attacks from scratch
- •atkgen drives an attack model adaptively rather than replaying a static prompt list
- •Backend-agnostic: the same scan runs against OpenAI, Bedrock, NIM or a local GGUF build
- •JSONL reports and hit logs make it practical as a CI gate, not just a manual audit
Who It’s For
Best For
- ✓AI engineers who must security-test an LLM feature before it ships
- ✓Red teams and AppSec staff extending their practice to generative AI
- ✓Platform teams adding automated model safety checks to CI pipelines
- ✓Anyone evaluating open-weight models for deployment risk
Prerequisites
- •Python 3.11-3.13 and comfort installing packages and running CLI tools
- •API keys or local weights for whichever model you intend to scan
- •Basic familiarity with prompt injection and jailbreak concepts
FAQ
What is garak — NVIDIA's LLM Vulnerability Scanner?
garak is NVIDIA's open-source scanner for probing language models for real security failures — prompt injection, jailbreaks, training-data leakage, toxic output, malware generation and package hallucination — rather than measuring benchmark accuracy. It is for engineers who have to sign off that a model or chatbot is safe to ship. After a run you get a timestamped JSONL report and a hit log naming exactly which attacks landed, which is what makes it usable as a release gate in CI rather than a one-off audit.
Is garak — NVIDIA's LLM Vulnerability Scanner free?
garak — NVIDIA's LLM Vulnerability Scanner is free to access.
What level is garak — NVIDIA's LLM Vulnerability Scanner for?
garak — NVIDIA's LLM Vulnerability Scanner is aimed at a intermediate audience. Recommended background: Python 3.11-3.13 and comfort installing packages and running CLI tools, API keys or local weights for whichever model you intend to scan, Basic familiarity with prompt injection and jailbreak concepts.
How long does garak — NVIDIA's LLM Vulnerability Scanner take?
Expect roughly ~1 hour to your first scan; a full default probe sweep runs for hours. Most learners work through it at their own pace.
What will I learn from garak — NVIDIA's LLM Vulnerability Scanner?
You'll learn: Run a first scan against an OpenAI, Hugging Face or REST-hosted target; Select probe families deliberately instead of always running the full default sweep; Understand how probes, detectors, generators, harnesses and evaluators fit together; Test for prompt injection, jailbreaks, encoding attacks and glitch tokens; Detect training-data leakage and package-hallucination risk in generated code; Read the JSONL report and hit log to separate real failures from noise; Write a custom probe or detector for a threat specific to your own application; Scan self-hosted GGUF and NIM deployments with the same suite as hosted APIs.
Topics
Sources
This page was written from 3 sources, 2 on domains other than github.com.