A Curriculum for Post-Training Agents
by Hugging Face
A five-step path from supervised fine-tuning to RL through real agent harnesses, mapped to the TRL trainer for each step.
Overview
A Curriculum for Post-Training Agents is a Hugging Face blog guide by Ben Burtenshaw, Sergio Paniego and Merve, published October 20, 2026, that doubles as the index for the team's Post-training Agents livestream series. It opens with a table mapping five steps to what the model learns from, the TRL trainer used and what you must prepare. Step 0 defines agent, harness and scaffold and links a glossary post, a 20-slide overview deck and the Hugging Face LLM Course for readers new to fine-tuning. Step 1 covers supervised fine-tuning on recorded agent traces with SFTTrainer, and is explicit that the model never sees its own mistakes. Step 2 covers distillation with DistillationTrainer, both from generated data and from per-token teacher probabilities, with distilled Qwen3 checkpoints in the linked collection. Step 3 explains GRPO with GRPOTrainer: sample several answers per prompt, score them, reinforce the ones above the group average, no separate value model. Step 4 moves to multi-turn RL environments built on OpenEnv, where rollouts become the bottleneck. Step 5 trains through an existing agent harness inside a remote sandbox with the experimental AsyncGRPOTrainer, which runs generation on a vLLM server and training on separate GPUs and syncs weights back. Each step links a livestream, scripts or a collection, and the matching TRL docs.
At a Glance
- Topic
- Fine-Tuning
- Level
- Intermediate
- Format
- Guide
- Cost
- Free
- Duration
- ~15 min read for the guide; the linked playlist adds four livestream sessions and two workshops, self-paced
- Provider
- Hugging Face
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓How supervised fine-tuning on recorded agent traces works with TRL's SFTTrainer, and why it cannot teach recovery from mistakes
- ✓How to distill a student model from a teacher using generated data or per-token probabilities with DistillationTrainer
- ✓How GRPO scores a group of sampled answers per prompt and reinforces those above the group average without a value model
- ✓How to move from single-turn reward functions to stateful multi-turn RL environments built on OpenEnv
- ✓How to train a model through an existing agent harness inside a remote sandbox so it learns where it will run
- ✓How AsyncGRPOTrainer separates vLLM rollout generation from training on different GPUs and syncs updated weights back
- ✓What data each step needs: trace datasets, prompts plus a teacher, reward functions, tasks plus environments
Highlights
- •Written by the Hugging Face engineers who maintain TRL and OpenEnv, and current as of October 2026
- •One table maps every post-training step to the exact TRL trainer and the inputs you have to prepare
- •Each step links a recorded livestream plus runnable scripts or a model and dataset collection, so it is more than a reading list
- •Says plainly what each method cannot do (SFT never sees its own errors, distillation is capped by the teacher, single-turn RL needs many attempts), which explains why the next step exists
- •TRL itself is a widely used open-source post-training library with about 19.6k GitHub stars
Who It’s For
Best For
- ✓ML engineers who have fine-tuned an LLM and now want to train a tool-using or coding agent
- ✓Teams deciding whether SFT, distillation or RL fits their agent's training data and compute budget
- ✓Researchers who want a map of TRL's agent-training features before reading the trainer docs
Prerequisites
- •Working Python and experience fine-tuning a transformer model (the guide points newcomers to the Hugging Face LLM Course first)
- •Basic agent concepts: tool calls, multi-turn interaction and harnesses
- •Some reinforcement learning intuition for steps 3 to 5
- •Access to GPUs, and for steps 4 and 5 multi-GPU or sandboxed compute plus vLLM
FAQ
What is A Curriculum for Post-Training Agents?
A free Hugging Face guide, published October 20, 2026, that lays out how to post-train an LLM to act as a tool-using agent. It is written for ML engineers who already fine-tune models and want to move to agent traces, distillation, GRPO, RL environments and harness-based training with TRL, OpenEnv and vLLM.
Is A Curriculum for Post-Training Agents free?
A Curriculum for Post-Training Agents is free to access.
What level is A Curriculum for Post-Training Agents for?
A Curriculum for Post-Training Agents is aimed at a intermediate audience. Recommended background: Working Python and experience fine-tuning a transformer model (the guide points newcomers to the Hugging Face LLM Course first), Basic agent concepts: tool calls, multi-turn interaction and harnesses, Some reinforcement learning intuition for steps 3 to 5, Access to GPUs, and for steps 4 and 5 multi-GPU or sandboxed compute plus vLLM.
How long does A Curriculum for Post-Training Agents take?
Expect roughly ~15 min read for the guide; the linked playlist adds four livestream sessions and two workshops, self-paced. Most learners work through it at their own pace.
What will I learn from A Curriculum for Post-Training Agents?
You'll learn: How supervised fine-tuning on recorded agent traces works with TRL's SFTTrainer, and why it cannot teach recovery from mistakes; How to distill a student model from a teacher using generated data or per-token probabilities with DistillationTrainer; How GRPO scores a group of sampled answers per prompt and reinforces those above the group average without a value model; How to move from single-turn reward functions to stateful multi-turn RL environments built on OpenEnv; How to train a model through an existing agent harness inside a remote sandbox so it learns where it will run; How AsyncGRPOTrainer separates vLLM rollout generation from training on different GPUs and syncs updated weights back; What data each step needs: trace datasets, prompts plus a teacher, reward functions, tasks plus environments.
Topics
Sources
This page was written from 3 sources, 1 on domains other than huggingface.co.