CS224N: NLP with Deep Learning
by Stanford University
Stanford's flagship NLP course, rewritten around LLMs — public slides, notes, assignments and lecture video.
Overview
The Winter 2026 offering is taught by Diyi Yang and Yejin Choi, and the site describes the course as a "thorough introduction to cutting-edge neural networks for NLP", covering deep learning fundamentals alongside current large language model research, with all implementation in PyTorch. Assessment is 48% assignments and 49% final project. The four assignments build in sequence: introduction to word vectors (6%), neural network foundations including tensor derivatives and dependency parsing (14%), self-attention and Transformers (14%), and large language model benchmarking and evaluation (14%). The default final project has students implement a minimalist version of GPT-2 and apply it to three downstream tasks; a custom project option is also offered, graded across proposal, milestone, poster and report. The ten-week lecture schedule reflects how far the syllabus has moved from its word-embeddings origins: history of NLP and word vectors, backpropagation, language models and RNNs, Transformers, pretraining covering scaling, systems and data, post-training with RLHF, SFT and DPO, efficient adaptation through prompting and PEFT, agents, tool use and RAG, benchmarking and evaluation, two lectures on reasoning, then guest lectures on tokenization and multilinguality, interpretability, social and broader impacts, multimodality and LoRA. Prerequisites are genuinely enforced: Python with NumPy and PyTorch, college calculus and linear algebra, probability and statistics, and prior machine learning coursework.
At a Glance
- Topic
- ML
- Level
- Advanced
- Format
- Course
- Cost
- Free
- Duration
- 10-week quarter; four assignments plus a final project, roughly 10-15 hours per week if followed fully
- Provider
- Stanford University
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Derive and implement word vector methods, then explain what they can and cannot capture
- ✓Work through backpropagation and tensor derivatives by hand before relying on autograd
- ✓Implement self-attention and a full Transformer architecture from scratch in PyTorch
- ✓Implement a minimalist GPT-2 and apply it to three distinct downstream tasks
- ✓Understand pretraining at scale — the data, systems and scaling-law considerations behind it
- ✓Compare post-training methods including supervised fine-tuning, RLHF and DPO
- ✓Apply efficient adaptation techniques such as prompting, PEFT and LoRA to a pretrained model
- ✓Design benchmarking and evaluation for language models, including where benchmarks mislead
Highlights
- •Course materials, assignments and lecture video are published openly every year — no enrollment required to follow along
- •The syllabus is genuinely current: pretraining scaling, RLHF/SFT/DPO, agents, tool use, RAG, interpretability and multimodality
- •The default final project (build a minimalist GPT-2, apply to three tasks) is a portfolio piece on its own
- •Taught by Diyi Yang and Yejin Choi, both active researchers, with guest lectures from specialists per topic
- •Assignment 4 is dedicated to LLM benchmarking and evaluation — still rare in a university curriculum
Who It’s For
Best For
- ✓Engineers who want research-grade depth rather than an applied API tutorial
- ✓ML practitioners preparing to read and implement NLP papers directly
- ✓Anyone intending to implement a transformer from scratch rather than call one
- ✓Graduate-level self-learners with the math background to keep up with assignments
Prerequisites
- •Proficiency in Python, specifically NumPy and PyTorch
- •College-level calculus and linear algebra, plus basic probability and statistics
- •Foundations of machine learning equivalent to Stanford CS221, CS229, CS230 or CS124
FAQ
What is CS224N: NLP with Deep Learning?
CS224N is the reference university course for deep learning in natural language processing, and the current offering has been rebuilt around large language models. Lectures, notes and assignments are published openly each year, so a self-directed engineer can follow the whole sequence — word vectors through pretraining, post-training, agents and evaluation — and finish by implementing a minimal GPT-2 from scratch.
Is CS224N: NLP with Deep Learning free?
CS224N: NLP with Deep Learning is free to access.
What level is CS224N: NLP with Deep Learning for?
CS224N: NLP with Deep Learning is aimed at a advanced audience. Recommended background: Proficiency in Python, specifically NumPy and PyTorch, College-level calculus and linear algebra, plus basic probability and statistics, Foundations of machine learning equivalent to Stanford CS221, CS229, CS230 or CS124.
How long does CS224N: NLP with Deep Learning take?
Expect roughly 10-week quarter; four assignments plus a final project, roughly 10-15 hours per week if followed fully. Most learners work through it at their own pace.
What will I learn from CS224N: NLP with Deep Learning?
You'll learn: Derive and implement word vector methods, then explain what they can and cannot capture; Work through backpropagation and tensor derivatives by hand before relying on autograd; Implement self-attention and a full Transformer architecture from scratch in PyTorch; Implement a minimalist GPT-2 and apply it to three distinct downstream tasks; Understand pretraining at scale — the data, systems and scaling-law considerations behind it; Compare post-training methods including supervised fine-tuning, RLHF and DPO; Apply efficient adaptation techniques such as prompting, PEFT and LoRA to a pretrained model; Design benchmarking and evaluation for language models, including where benchmarks mislead.
Topics
Sources
This page was written from 2 sources, 1 on domains other than web.stanford.edu.