ModelsMLFine-TuningRAG

CME 295: Transformers & Large Language Models

by Stanford University

IntermediateCourseFree9 lectures of ~2 hours across a 10-week quarter; the free cheatsheet is a ~45-minute read

Stanford's quarter-long tour from word2vec to GRPO, with a free 250-page illustrated textbook and a printable cheatsheet.

Start LearningAdded Aug 30, 2026 · Updated Aug 30, 2026

Overview

CME 295 is taught by Afshine and Shervine Amidi, the authors of the widely circulated Stanford CS229 and CS230 cheatsheets, and was offered in Autumn 2025-2026 on Fridays in Thornton 110. The nine-lecture arc is unusually complete for a single quarter. Lecture 1 covers NLP background and tasks, tokenization, embeddings, word2vec, RNNs, LSTMs, the attention mechanism and the transformer architecture. Lecture 2 goes into attention approximation, multi-head versus multi-query versus grouped-query attention, position embeddings including RoPE, and encoder models such as BERT. Lecture 3 defines large language models proper and covers mixture of experts, context management, sampling strategies and prompting including chain of thought. Lecture 4 handles training: pretraining, quantization, hardware optimisation, supervised finetuning and parameter-efficient methods such as LoRA. After the midterm, Lecture 5 covers preference tuning — RLHF, reward modelling, PPO variants and DPO — and Lecture 6 covers reasoning models, GRPO and scaling considerations. Lecture 7 is retrieval-augmented generation, advanced RAG, function calling, agents and ReAct; Lecture 8 is evaluation with LLM-as-a-judge, best practices and its characteristic biases; Lecture 9 recaps and surveys emerging trends. Assessment is a midterm and a final with no programming homework, which is worth knowing before you enrol expecting a build-along course. The free public artifacts are the syllabus, an illustrated VIP cheatsheet published on GitHub, and the companion textbook Super Study Guide: Transformers & Large Language Models — roughly 250 pages and 600 illustrations, readable free at superstudy.guide.

At a Glance

Topic: Models
Level: Intermediate
Format: Course
Cost: Free
Duration: 9 lectures of ~2 hours across a 10-week quarter; the free cheatsheet is a ~45-minute read
Provider: Stanford University
Hands-on: No
Certificate: None

What You’ll Learn

✓Trace NLP from word2vec and LSTMs through attention to the full transformer architecture
✓Compare multi-head, multi-query and grouped-query attention and RoPE position embeddings
✓Explain mixture of experts, context management and sampling strategies in modern LLMs
✓Apply quantization, LoRA and parameter-efficient methods on top of supervised finetuning
✓Work through preference tuning: reward modelling, PPO variants, DPO and GRPO
✓Reason about retrieval-augmented generation, function calling and ReAct-style agents
✓Design LLM-as-a-judge evaluations and recognise the biases they characteristically introduce

Highlights

•Free companion textbook of roughly 250 pages and 600 illustrations at superstudy.guide
•A printable VIP cheatsheet on GitHub distils the whole quarter into a few pages
•Goes past the 2017 transformer into DPO, GRPO and reasoning-model training
•Taught by the Amidi brothers, whose CS229 and CS230 cheatsheets are a Stanford staple
•One coherent syllabus covering architecture, training, retrieval, agents and evaluation together

Who It’s For

Best For

✓Engineers who want the theory behind the LLM stack they already ship
✓ML practitioners preparing for interviews on transformers and post-training
✓Self-learners who prefer one structured syllabus over scattered blog posts
✓Anyone who needs a reliable reference sheet for attention and finetuning variants

Prerequisites

•Calculus and linear algebra
•Basic machine learning concepts such as loss functions, gradient descent and neural networks
•No programming assignments are set, so no framework experience is strictly required

FAQ

What is CME 295: Transformers & Large Language Models?

A Stanford graduate course that walks the full arc of modern language modelling — tokenization and attention, transformer variants, pretraining and quantization, LoRA finetuning, RLHF and DPO preference tuning, reasoning-model RL, RAG, agents and LLM-as-a-judge evaluation. Written for engineers who already use LLMs and want the theory underneath them. The syllabus, cheatsheet and companion textbook are free online.

Is CME 295: Transformers & Large Language Models free?

CME 295: Transformers & Large Language Models is free to access.

What level is CME 295: Transformers & Large Language Models for?

CME 295: Transformers & Large Language Models is aimed at a intermediate audience. Recommended background: Calculus and linear algebra, Basic machine learning concepts such as loss functions, gradient descent and neural networks, No programming assignments are set, so no framework experience is strictly required.

How long does CME 295: Transformers & Large Language Models take?

Expect roughly 9 lectures of ~2 hours across a 10-week quarter; the free cheatsheet is a ~45-minute read. Most learners work through it at their own pace.

What will I learn from CME 295: Transformers & Large Language Models?

You'll learn: Trace NLP from word2vec and LSTMs through attention to the full transformer architecture; Compare multi-head, multi-query and grouped-query attention and RoPE position embeddings; Explain mixture of experts, context management and sampling strategies in modern LLMs; Apply quantization, LoRA and parameter-efficient methods on top of supervised finetuning; Work through preference tuning: reward modelling, PPO variants, DPO and GRPO; Reason about retrieval-augmented generation, function calling and ReAct-style agents; Design LLM-as-a-judge evaluations and recognise the biases they characteristically introduce.

Topics

transformerslarge language modelsStanfordattentionpost-training

Sources

This page was written from 3 sources, 1 on domains other than cme295.stanford.edu.