Post-training of LLMs
by DeepLearning.AI
SFT, DPO and online RL, each explained then coded, in under ninety minutes.
Overview
Post-training of LLMs is a DeepLearning.AI short course taught by Banghua Zhu, who took his PhD at UC Berkeley under Jiantao Jiao and Michael I. Jordan, co-founded Nexusflow, built the Starling-7B and Athene-V2-72B open models, and now leads a foundation-model and reinforcement-learning lab at the University of Washington alongside work on the SGLang serving stack. The course runs 1 hour 26 minutes across nine lessons and is deliberately structured as three concept-then-code pairs, one per post-training family: an introduction to post-training, basics of supervised fine-tuning followed by SFT in practice with a code example, basics of Direct Preference Optimization followed by DPO in practice, and basics of online reinforcement learning followed by online RL in practice, closing with a conclusion. The hands-on notebooks pull models from HuggingFace and walk through three distinct outcomes: turning a base model into an instruction-following model with SFT, changing an assistant's identity and behaviour through preference pairs with DPO, and improving mathematical capability with an online RL loop. Lessons are free to watch; the graded assignment and certificate of accomplishment require a paid DeepLearning.AI Pro membership. The emphasis throughout is method selection - knowing which of the three techniques fixes which failure mode - rather than large-scale training engineering.
At a Glance
- Topic
- Fine-Tuning
- Level
- Intermediate
- Format
- Course
- Cost
- Free
- Duration
- 1 hour 26 minutes across 9 lessons, self-paced
- Provider
- DeepLearning.AI
- Hands-on
- Yes — code/exercises
- Certificate
- Available
What You’ll Learn
- ✓Choose between SFT, DPO and online RL for a given goal
- ✓Convert a base model into an instruction-following model using SFT
- ✓Build preference pairs and change assistant identity through DPO
- ✓Run an online reinforcement-learning loop to improve mathematical reasoning
- ✓Load and swap HuggingFace checkpoints inside each hands-on notebook
- ✓Recognise which post-training method addresses which observed failure mode
- ✓Judge when post-training beats prompting or retrieval for a problem
Highlights
- •Every method gets both a concept lesson and a separate runnable code lesson
- •Taught by the researcher behind Starling-7B, Athene-V2 and the SGLang stack
- •Under ninety minutes end to end, so it fits into a single sitting
- •Free to watch, with a graded assignment available to Pro members
- •Covers online RL, which most introductory fine-tuning courses skip entirely
Who It’s For
Best For
- ✓ML engineers about to attempt their first fine-tune
- ✓Practitioners deciding between DPO and online reinforcement learning
- ✓Backend developers moving from prompting into model training
Prerequisites
- •Working Python with basic PyTorch or HuggingFace experience
- •Understanding of what pretraining produces and what instruction tuning is
FAQ
What is Post-training of LLMs?
A short DeepLearning.AI course that walks through the three main post-training methods - supervised fine-tuning, Direct Preference Optimization, and online reinforcement learning - pairing a concept lesson with a runnable code lesson for each. It is built for engineers who understand what pretraining produces and now need to decide which post-training technique fixes the failure mode in front of them.
Is Post-training of LLMs free?
Post-training of LLMs is free to access.
What level is Post-training of LLMs for?
Post-training of LLMs is aimed at a intermediate audience. Recommended background: Working Python with basic PyTorch or HuggingFace experience, Understanding of what pretraining produces and what instruction tuning is.
How long does Post-training of LLMs take?
Expect roughly 1 hour 26 minutes across 9 lessons, self-paced. Most learners work through it at their own pace.
What will I learn from Post-training of LLMs?
You'll learn: Choose between SFT, DPO and online RL for a given goal; Convert a base model into an instruction-following model using SFT; Build preference pairs and change assistant identity through DPO; Run an online reinforcement-learning loop to improve mathematical reasoning; Load and swap HuggingFace checkpoints inside each hands-on notebook; Recognise which post-training method addresses which observed failure mode; Judge when post-training beats prompting or retrieval for a problem.
Topics
Sources
This page was written from 2 sources, 1 on domains other than deeplearning.ai.