Modern LLM Notebook
by WalkingLabs
A free notebook course that builds an LLM stack by hand, from BPE and GPT to MoE, DPO, speculative decoding and vLLM serving.
Overview
Modern LLM Notebook is an open GitHub course maintained by WalkingLabs under a CC BY-NC-SA 4.0 license, with Chinese source notebooks and an English mirror in notebooks-en plus an online reader on GitHub Pages. It launched as bilingual notebooks in May 2026, expanded MLA, distributed training, quantization and tool calling in June, was reorganized into four parts with rebuilt inference labs in August, and added from-scratch training labs in September 2026. Part 1, Foundation, covers tokenizers, BPE, embeddings, position encoding, attention blocks, a GPT built from scratch and a BERT encoder. Part 2, Training, covers modern architecture changes (RMSNorm, SwiGLU, RoPE, GQA, QK-Norm, MLA), loss and gradients, pretraining and SFT with chat templates, KV cache variants, distributed training with Accelerate, ZeRO and Megatron-LM, MoE routing, scaling laws, data engineering, LoRA, distillation, function calling and RLHF with PPO and DPO. Part 3, Inference, covers decoding, FlashAttention and PagedAttention, GPTQ and AWQ quantization, speculative decoding, continuous batching, evaluation and serving with vLLM and SGLang. Part 4 covers long context with YaRN, reasoning models, VLMs, linear attention and Mamba, and on-policy distillation. The repo ships a 61.55M-parameter DotLM trained on one AMD MI300X with reported benchmark results.
At a Glance
- Topic
- ML
- Level
- Intermediate
- Format
- Course
- Cost
- Free
- Duration
- Self-paced, 31 notebooks plus appendices; several weeks part-time (no official estimate)
- Provider
- WalkingLabs
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓How to implement a BPE tokenizer with merge rules, encoding and decoding from scratch
- ✓How to build a decoder-only GPT and a MiniBERT encoder from attention blocks in PyTorch
- ✓How RMSNorm, SwiGLU, RoPE, GQA, QK-Norm and MLA change the original transformer design
- ✓How to write a pretraining and SFT loop with chat templates and correct label shifting
- ✓How MoE routers pick top-k experts and how auxiliary-loss-free load balancing works
- ✓How to implement a LoRA linear layer, merge adapters and run logit distillation
- ✓How reward models, PPO and DPO fit together in preference alignment
- ✓How KV cache, PagedAttention, GPTQ and AWQ quantization and speculative decoding speed up inference
- ✓How to estimate FLOPs, GPU hours and memory from Kaplan and Chinchilla scaling laws
Highlights
- •Covers training, inference and serving in one sequence of 31 notebooks, rare for a free course
- •Most notebooks run on CPU with 16 GB of RAM; Colab, ModelScope and an AMD cloud template are offered for GPU work
- •Ships a real end-to-end artifact: a 61.55M-parameter DotLM pretrained and SFT-tuned with published validation perplexity and benchmark scores
- •Actively maintained through 2026, with monthly changelog entries from May to September and about 290 GitHub stars
- •The Chinese edition is the source; the English mirror is updated alongside it, so some English notebooks may lag
Who It’s For
Best For
- ✓Software engineers moving into LLM training or inference work who want to implement each component
- ✓ML engineers preparing for model-internals interviews on KV cache, MoE, quantization and RLHF
- ✓Self-learners who finished a GPT-from-scratch tutorial and want current architectures and serving
- ✓Instructors looking for openly licensed notebooks to adapt for a non-commercial LLM course
Prerequisites
- •Basic Python and matrix operations, per the course's own stated starting point
- •Local setup with Python 3.9+, PyTorch 2.0+, NumPy and Jupyter, or a Colab account
- •Helpful but not required: prior exposure to neural network training and backpropagation
FAQ
What is Modern LLM Notebook?
Modern LLM Notebook is a free, bilingual (Chinese and English) course of 31 Jupyter notebooks from WalkingLabs that implements the core pieces of modern large language models in PyTorch. It is for engineers who know Python and want to understand training and inference internals by writing tokenizers, attention, KV caches, LoRA, DPO and quantization themselves, then train a small model end to end.
Is Modern LLM Notebook free?
Modern LLM Notebook is free to access.
What level is Modern LLM Notebook for?
Modern LLM Notebook is aimed at a intermediate audience. Recommended background: Basic Python and matrix operations, per the course's own stated starting point, Local setup with Python 3.9+, PyTorch 2.0+, NumPy and Jupyter, or a Colab account, Helpful but not required: prior exposure to neural network training and backpropagation.
How long does Modern LLM Notebook take?
Expect roughly Self-paced, 31 notebooks plus appendices; several weeks part-time (no official estimate). Most learners work through it at their own pace.
What will I learn from Modern LLM Notebook?
You'll learn: How to implement a BPE tokenizer with merge rules, encoding and decoding from scratch; How to build a decoder-only GPT and a MiniBERT encoder from attention blocks in PyTorch; How RMSNorm, SwiGLU, RoPE, GQA, QK-Norm and MLA change the original transformer design; How to write a pretraining and SFT loop with chat templates and correct label shifting; How MoE routers pick top-k experts and how auxiliary-loss-free load balancing works; How to implement a LoRA linear layer, merge adapters and run logit distillation; How reward models, PPO and DPO fit together in preference alignment; How KV cache, PagedAttention, GPTQ and AWQ quantization and speculative decoding speed up inference; How to estimate FLOPs, GPU hours and memory from Kaplan and Chinchilla scaling laws.
Topics
Sources
This page was written from 2 sources, 1 on domains other than github.com.