Reinforcement Learning from Human Feedback: Post-Training Course
by Nathan Lambert
Nathan Lambert's free video lecture series on LLM post-training: reward models, RL, DPO, reasoning models and agents.
Overview
The Post-Training Course is Nathan Lambert's companion video series to his open RLHF Book (rlhfbook.com), which Manning published in print in 2026. Each lecture maps to specific book chapters: Lecture 0 covers the ML foundations of LLM post-training; Lecture 1 is an overview (chapters 1-3); Lecture 2 covers instruction fine-tuning, reward models and rejection sampling (chapters 4, 5, 9); Lectures 3 and 4 cover RL motivation and math, then RL implementation and practice (chapter 6); Lecture 5 is the rise of reasoning models (chapter 7); Lecture 6 is Direct Preference Optimization (chapter 8); Lecture 7 covers synthetic data and modern post-training methods (chapter 12); Lecture 8 is preferences and preference data (chapters 10-11); Lecture 9 is over-optimization and RLHF's bad reputation (chapter 14); Lecture 10 covers regularization in RL, why RL generalizes and why SFT forgets (chapter 15); Lecture 11 is tool use, function calling and the road to agents (chapter 13); Lecture 12 is the evolution of frontier model evaluation (chapter 16); and Lecture 13 introduces character training (chapter 17). Three Q&A sessions, a conversation on frontier post-training recipes in 2026 and a SALA 2026 invited talk round it out. Slides and sources are published, and the book's MIT-licensed GitHub repository (about 2.4k stars) contains reference code for SFT, PPO, REINFORCE, GRPO, RLOO, reward-model training, DPO variants, rejection sampling and distillation. Lambert, a post-training researcher, states it is aimed at early AI PhD or master's students but accessible to anyone willing to work.
At a Glance
- Topic
- Fine-Tuning
- Level
- Intermediate
- Format
- Video
- Cost
- Free
- Duration
- Lecture 0 plus 13 video lectures, 3 Q&A sessions and bonus talks, self-paced; lecture lengths vary
- Provider
- Nathan Lambert
- Hands-on
- No
- Certificate
- None
What You’ll Learn
- ✓How instruction fine-tuning, reward models and rejection sampling fit into post-training pipelines
- ✓The math behind policy-gradient RL for LLMs and how it is implemented in practice
- ✓How reasoning models are trained with RL on verifiable rewards
- ✓How Direct Preference Optimization works and how it compares with RL-based RLHF
- ✓How synthetic data and preference data are collected and used in modern recipes
- ✓Why over-optimization happens and how regularization affects generalization versus forgetting
- ✓How post-training enables tool use, function calling and agentic behaviour
- ✓How frontier model evaluation has evolved and what character training involves
Highlights
- •Every lecture maps to named chapters of the open RLHF Book, so video and text reinforce each other
- •Companion GitHub repo (MIT code, ~2.4k stars) has reference implementations of PPO, GRPO, RLOO, DPO variants and reward models
- •Includes 2026-current material: frontier post-training recipes conversation, agents and tool-use lecture, evaluation lecture
- •Completely free, with slides and sources published alongside the videos
Who It’s For
Best For
- ✓ML engineers fine-tuning or aligning open-weight LLMs
- ✓Early graduate students starting research in RLHF and post-training
- ✓AI engineers who want to understand how reasoning and agentic models are trained
Prerequisites
- •Solid Python and working knowledge of deep learning
- •Basic familiarity with language models; no prior RL background required per the author
FAQ
What is Reinforcement Learning from Human Feedback: Post-Training Course?
A free video lecture course on LLM post-training by Nathan Lambert, the author of the RLHF Book. It is for engineers and early graduate students who want to understand how instruction tuning, reward models, RL, DPO, synthetic data and reasoning-model training fit together, and to read or modify the book's reference implementations of those methods.
Is Reinforcement Learning from Human Feedback: Post-Training Course free?
Reinforcement Learning from Human Feedback: Post-Training Course is free to access.
What level is Reinforcement Learning from Human Feedback: Post-Training Course for?
Reinforcement Learning from Human Feedback: Post-Training Course is aimed at a intermediate audience. Recommended background: Solid Python and working knowledge of deep learning, Basic familiarity with language models; no prior RL background required per the author.
How long does Reinforcement Learning from Human Feedback: Post-Training Course take?
Expect roughly Lecture 0 plus 13 video lectures, 3 Q&A sessions and bonus talks, self-paced; lecture lengths vary. Most learners work through it at their own pace.
What will I learn from Reinforcement Learning from Human Feedback: Post-Training Course?
You'll learn: How instruction fine-tuning, reward models and rejection sampling fit into post-training pipelines; The math behind policy-gradient RL for LLMs and how it is implemented in practice; How reasoning models are trained with RL on verifiable rewards; How Direct Preference Optimization works and how it compares with RL-based RLHF; How synthetic data and preference data are collected and used in modern recipes; Why over-optimization happens and how regularization affects generalization versus forgetting; How post-training enables tool use, function calling and agentic behaviour; How frontier model evaluation has evolved and what character training involves.
Topics
Sources
This page was written from 2 sources, 1 on domains other than rlhfbook.com.