CS 294-288: Data-Centric LLMs (UC Berkeley, Fall 2026)
by UC Berkeley (Sewon Min)
A curated, week-by-week reading list on where LLM training data comes from, how it scales and what it costs
Overview
CS 294-288: Data-Centric LLMs is a special-topics seminar at UC Berkeley taught by Sewon Min in Fall 2026, running August 26 to December 11 on Tuesdays and Thursdays. The Berkeley catalog describes it as a course on where LLM training datasets come from, how they are used, why they work and what problems they create, aimed mainly at PhD students and built around paper readings, discussions and an open-ended project. The public course page lists each session with prerequisite, main and optional papers. Early weeks cover pretraining data curation (GPT-3, DataComp-LM, FineWeb, Dolma, deduplication), a guest lecture by Shayne Longpre on the data ecosystem, scaling laws including Chinchilla and mixture-of-experts scaling, and scaling under data constraints and infinite compute. Later sessions cover data provenance and representation, copyright and consent (Foundation Models and Fair Use, Consent in Crisis, the Common Pile), synthetic data for post-training and reasoning (LIMA, Spurious Rewards), synthetic pretraining (Rephrasing the Web, BeyondWeb, Nemotron-CC), model collapse, frontier open model reports such as DeepSeek-V3 and DeepSeek-R1, AI-generated text detection, training data attribution and valuation, membership inference and training data extraction. Some slides are public as PDFs. Students are expected to know CS 288 material, and the CS 288 Spring 2026 site is linked as background.
At a Glance
- Topic
- Models
- Level
- Advanced
- Format
- Course
- Cost
- Free
- Duration
- One semester (Aug 26 to Dec 11, 2026), two 80-minute sessions a week; self-study time depends on how many of the assigned papers you read
- Provider
- UC Berkeley (Sewon Min)
- Hands-on
- No
- Certificate
- None
What You’ll Learn
- ✓How pretraining corpora such as FineWeb, DataComp-LM and Dolma are filtered, deduplicated and curated
- ✓How compute-optimal scaling laws, including Chinchilla and mixture-of-experts variants, guide data and model sizing
- ✓What happens to scaling when data runs out, and how data filtering interacts with compute budgets
- ✓How synthetic data is used for post-training, reasoning RL and rephrased synthetic pretraining at scale
- ✓Why recursive training on model outputs can cause model collapse and loss of linguistic diversity
- ✓How copyright, consent and data provenance research is changing what web data can be used for training
- ✓How training data attribution and influence methods estimate which examples shaped a model's output
- ✓How membership inference and training data extraction attacks work, and where their claims break down
Highlights
- •Every session lists prerequisite, main and optional papers, so the schedule works as a ready-made self-study reading list
- •Current to Fall 2026, with recent material such as DeepSeek-V4, GLM-5, BeyondWeb and Nemotron-CC
- •Taught by Sewon Min, whose research covers LLM training data, and includes a guest lecture from Shayne Longpre on the data ecosystem
- •Pairs technical topics (scaling, filtering, synthetic data) with legal and social ones (fair use, consent, detection) that engineering courses rarely cover
- •Introduction, pretraining data curation and guest lecture slides are publicly linked
Who It’s For
Best For
- ✓ML engineers and researchers who build or curate pretraining and fine-tuning datasets
- ✓PhD students looking for a structured reading list on LLM data research
- ✓Practitioners evaluating synthetic data pipelines or data licensing risk for model training
- ✓Readers who already know transformers and want to understand why data choices drive model quality
Prerequisites
- •A strong background in ML, NLP or LLMs, equivalent to Berkeley's CS 288
- •Comfort reading research papers independently, since there are no recorded lectures on the page
- •Familiarity with transformer pretraining and fine-tuning
FAQ
What is CS 294-288: Data-Centric LLMs (UC Berkeley, Fall 2026)?
CS 294-288 is a Fall 2026 UC Berkeley graduate seminar taught by Sewon Min on the data behind large language models. Its public schedule is a reading list for researchers and senior ML engineers who want to understand pretraining data curation, scaling laws, synthetic data, copyright, model collapse and training data attribution from the primary papers.
Is CS 294-288: Data-Centric LLMs (UC Berkeley, Fall 2026) free?
CS 294-288: Data-Centric LLMs (UC Berkeley, Fall 2026) is free to access.
What level is CS 294-288: Data-Centric LLMs (UC Berkeley, Fall 2026) for?
CS 294-288: Data-Centric LLMs (UC Berkeley, Fall 2026) is aimed at a advanced audience. Recommended background: A strong background in ML, NLP or LLMs, equivalent to Berkeley's CS 288, Comfort reading research papers independently, since there are no recorded lectures on the page, Familiarity with transformer pretraining and fine-tuning.
How long does CS 294-288: Data-Centric LLMs (UC Berkeley, Fall 2026) take?
Expect roughly One semester (Aug 26 to Dec 11, 2026), two 80-minute sessions a week; self-study time depends on how many of the assigned papers you read. Most learners work through it at their own pace.
What will I learn from CS 294-288: Data-Centric LLMs (UC Berkeley, Fall 2026)?
You'll learn: How pretraining corpora such as FineWeb, DataComp-LM and Dolma are filtered, deduplicated and curated; How compute-optimal scaling laws, including Chinchilla and mixture-of-experts variants, guide data and model sizing; What happens to scaling when data runs out, and how data filtering interacts with compute budgets; How synthetic data is used for post-training, reasoning RL and rephrased synthetic pretraining at scale; Why recursive training on model outputs can cause model collapse and loss of linguistic diversity; How copyright, consent and data provenance research is changing what web data can be used for training; How training data attribution and influence methods estimate which examples shaped a model's output; How membership inference and training data extraction attacks work, and where their claims break down.
Topics
Sources
This page was written from 2 sources, 1 on domains other than sewonmin.com.