CMU 11-711: Advanced Natural Language Processing (Spring 2026)

11-711 is Carnegie Mellon's graduate Advanced NLP course, run in Spring 2026 by Sean Welleck of the Language Technologies Institute with eight teaching assistants, and it is the closest thing available for free to a modern LLM systems curriculum taught at research depth. Twenty-three lectures run Tuesday and Thursday from January to April in a deliberate order: introduction and fundamentals, learned representations, autoregressive language modeling, recurrent neural networks, attention and Transformers, pretraining, scaling laws and in-context learning, fine-tuning and distillation, decoding algorithms, retrieval and RAG (guest lecture by Akari Asai), two multimodal sessions, evaluation techniques, research skills and experimental design, diffusion and flows, two reinforcement learning lectures, language model-based agents, quantization, parallelism and distributed training, mixture of experts, scaling sequence length, and test-time scaling. Four assignments carry the practical work: Assignment 1 is 'Build Your Own LLaMa', Assignment 2 an end-to-end NLP system build, Assignment 3 a literature review plus state-of-the-art reimplementation, and Assignment 4 an open-ended team research project worth 25% of the grade, ending in poster sessions. Slides and per-lecture code walkthroughs are published openly in the companion GitHub repository cmu-l3/anlp-spring2026-code, twenty numbered lecture directories with 125 stars, so a self-studier gets the full curriculum, readings and runnable examples without enrolling.

Topic

ML

Level

Advanced

Format

Course

Cost

Free

Duration

~15 weeks, 23 lectures (2 x 80 min/week) plus 4 assignments; self-paced from the public slides and code

Provider

Carnegie Mellon University — Language Technologies Institute

Hands-on

Yes — code/exercises

Certificate

None

• ✓Implement a LLaMa-style transformer from scratch rather than calling a library
• ✓Reason about scaling laws and in-context learning as engineering constraints, not trivia
• ✓Build and evaluate a retrieval-augmented pipeline using current RAG research methods
• ✓Apply reinforcement learning to language models, from fundamentals through test-time scaling
• ✓Quantize models and choose parallelism strategies for distributed training runs
• ✓Design a defensible evaluation for an NLP system and reproduce a published baseline
• ✓Take an open-ended research project from literature review through poster presentation

• •Slides plus twenty lecture code walkthroughs are public on GitHub (125 stars), so self-study is complete rather than partial
• •Assignment 1 is a from-scratch LLaMa implementation — the fastest cure for treating transformers as a black box
• •The 2026 syllabus is genuinely current: diffusion and flows, mixture of experts, agents and test-time scaling are full lectures, not appendices
• •Guest lecture on retrieval and RAG from Akari Asai, an active researcher in that area
• •Honest caveat: no lecture videos are posted for Spring 2026 — earlier 11-711 editions have public YouTube playlists if you want a talking head

This page was written from 2 sources, 1 on domains other than cmu-l3.github.io.

1. 1.cmu-l3.github.io — anlp spring2026vendor
2. 2.github.com — anlp spring2026 code