10-405/10-605/10-805: ML with Large Datasets (CMU, Fall 2026)
by Carnegie Mellon University (William W. Cohen)
CMU's large-scale ML course: Spark, distributed optimization, LSH, then LoRA, KV caching, compression, RAG and GPT-2 training.
Overview
10-405/10-605/10-805 ML with Large Datasets is taught at Carnegie Mellon by William W. Cohen, with education associate Jacob Rast and six TAs. The Fall 2026 offering runs 25 sessions on Mondays and Wednesdays, and CMU released the recordings on YouTube through its CS Learn program in November 2026. The first half covers classic scale: Map-Reduce and Spark, workflows for Map-Reduce systems, three lectures on learning as optimization (distributed learning, parameter mixing, all-reduce, sparsity, hash kernels and matrix factorization), randomized algorithms such as Bloom filters, Count-Min sketches, LSH, MinHash and SimHash, and autodiff with parallel optimization on GPUs. After the first exam the course turns to modern deep learning: architectures and transformers, LoRA, BPE and hyperparameter search, embarrassingly parallel training, model compression and pruning, KV caching for LLMs, contrastive learning and retrieval, RAG and transformer architecture optimization, and scalable evaluation. Guest lectures cover consistent LLM systems (Jay Yoon Lee) and the practical limits of deep learning (Leo Boytsov). The five homeworks are scalable entity resolution with Naive Bayes, parallel linear regression, locality-sensitive hashing, LoRA and GPT-2 training. Each is submitted twice: a human-only first attempt, then an AI-assisted correction round.
At a Glance
- Topic
- ML
- Level
- Intermediate
- Format
- Course
- Cost
- Free
- Duration
- 25 class sessions of 80 minutes (August 24 to December 2, 2026), recorded on YouTube, plus five homework assignments
- Provider
- Carnegie Mellon University (William W. Cohen)
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Write Map-Reduce and Spark workflows for preprocessing and learning on large datasets
- ✓Distribute model optimization with parameter mixing, all-reduce and hash kernels
- ✓Apply Bloom filters, Count-Min sketches, MinHash and SimHash to large-scale ML problems
- ✓Fine-tune transformer models with LoRA and run hyperparameter search efficiently
- ✓Compress and prune models and use KV caching to cut LLM inference cost
- ✓Build contrastive retrieval and retrieval-augmented generation pipelines over large corpora
- ✓Train a GPT-2 model end to end in the final programming assignment
- ✓Analyze the memory, computation and communication complexity of large-scale learning methods
Highlights
- •Fall 2026 lectures released free on YouTube by CMU CS Learn, part of a launch that drew 2,305 points on Hacker News (checked November 18, 2026)
- •Bridges classic big-data ML (Spark, sketches, LSH) with current LLM practice (LoRA, KV caching, RAG)
- •Homework uses a human-only attempt followed by an AI-assisted correction round
- •Hands-on assignments end with LoRA fine-tuning and GPT-2 training
- •Taught by William W. Cohen, with guest lectures on LLM consistency and deep learning robustness
Who It’s For
Best For
- ✓Data scientists and ML engineers whose datasets have outgrown a single machine
- ✓Engineers who want a structured path from distributed ML fundamentals to LLM efficiency techniques
- ✓Graduate students preparing for ML engineering or MLOps roles
Prerequisites
- •An introductory machine learning course (CMU lists 10-301, 10-315, 10-601, 10-701 or 10-715)
- •Strong programming skills; 15-210 or 15-214 or equivalent are suggested
- •Working Python
FAQ
What is 10-405/10-605/10-805: ML with Large Datasets (CMU, Fall 2026)?
ML with Large Datasets is William Cohen's Carnegie Mellon course on learning from data too big for one machine, with the Fall 2026 lectures released free on YouTube. It suits engineers who already know basic ML and want to scale it, moving from Map-Reduce and Spark to LoRA, KV caching, model compression, retrieval and GPT-2 training.
Is 10-405/10-605/10-805: ML with Large Datasets (CMU, Fall 2026) free?
10-405/10-605/10-805: ML with Large Datasets (CMU, Fall 2026) is free to access.
What level is 10-405/10-605/10-805: ML with Large Datasets (CMU, Fall 2026) for?
10-405/10-605/10-805: ML with Large Datasets (CMU, Fall 2026) is aimed at a intermediate audience. Recommended background: An introductory machine learning course (CMU lists 10-301, 10-315, 10-601, 10-701 or 10-715), Strong programming skills; 15-210 or 15-214 or equivalent are suggested, Working Python.
How long does 10-405/10-605/10-805: ML with Large Datasets (CMU, Fall 2026) take?
Expect roughly 25 class sessions of 80 minutes (August 24 to December 2, 2026), recorded on YouTube, plus five homework assignments. Most learners work through it at their own pace.
What will I learn from 10-405/10-605/10-805: ML with Large Datasets (CMU, Fall 2026)?
You'll learn: Write Map-Reduce and Spark workflows for preprocessing and learning on large datasets; Distribute model optimization with parameter mixing, all-reduce and hash kernels; Apply Bloom filters, Count-Min sketches, MinHash and SimHash to large-scale ML problems; Fine-tune transformer models with LoRA and run hyperparameter search efficiently; Compress and prune models and use KV caching to cut LLM inference cost; Build contrastive retrieval and retrieval-augmented generation pipelines over large corpora; Train a GPT-2 model end to end in the final programming assignment; Analyze the memory, computation and communication complexity of large-scale learning methods.
Topics
Sources
This page was written from 5 sources, 3 on domains other than 10605.github.io.