Scalable AI: Bridging Theory, Understanding, and Practice (UC Berkeley EE 290 / 194, Fall 2026)
by UC Berkeley
Follow a large language model from GPU performance limits through pre-training, post-training, and production inference.
Overview
Scalable AI is UC Berkeley's EE 290 / 194 course, taught in Fall 2026 by Anant Sahai and Jiantao Jiao with NVIDIA's Venkat Srinivasan on the teaching staff. It follows a large model through its whole life: architecture, pre-training, post-training, efficient inference, applications, and open research problems. The 28 class sessions open with performance analysis (roofline models, memory bottlenecks, token economics), then cover MoE, sparse and long-context architectures, parallelism, FlashAttention, and SonicMoE. The pre-training block works through tokenization, deduplication, filtering, data mixtures, optimizers, and a Nano-V3 case study. Post-training covers evaluation, supervised fine-tuning data, RL environments, and a Nemotron-NanoV3 case study. Inference sessions compare Dynamo, TensorRT-LLM, vLLM, and SGLang, the application block covers context engineering, harness design, and safety, and the course's tool list includes NeMo Guardrails and garak. Four group assignments cover performance, pre-training, post-training, and inference, and a research project is half the grade. Enrolled teams get one eight-GPU H100 node on GCP for the semester, which outside readers have to replace themselves. Lecture PDFs, readings, and the first two assignment archives were posted on the site when checked on November 11, 2026. A3 and A4 had no download link at that date, and Spring 2026 lecture videos required a Berkeley email. The stack leans heavily on NVIDIA's NeMo tools.
At a Glance
- Topic
- Models
- Level
- Advanced
- Format
- Course
- Cost
- Free
- Duration
- One semester (Aug 26 to Dec 18, 2026): 28 class sessions, about 42 hours of class time, plus four assignments of about 3-4 weeks each
- Provider
- UC Berkeley
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Model LLM performance limits with roofline analysis, memory bottlenecks, and token economics
- ✓Compare MoE, sparse, and long-context architectures under real hardware and serving constraints
- ✓Choose training parallelism strategies and study FlashAttention and SonicMoE in modern model stacks
- ✓Prepare pre-training data with deduplication, filtering, data mixtures, and NeMo Curator pipelines
- ✓Build supervised fine-tuning datasets with NeMo Data Designer and design RL environments
- ✓Run RL post-training with NeMo RL and NeMo Gym, following the Nemotron-NanoV3 case study
- ✓Compare Dynamo, TensorRT-LLM, vLLM, and SGLang for high-performance LLM inference in practice
- ✓Apply context engineering, harness design, deployment guardrails, and open-weight safety to LLM applications
Highlights
- •Covers one model lifecycle end to end, with Nano-V3 pre-training and Nemotron-NanoV3 post-training case studies
- •Lecture slide PDFs and assignment archives are posted on the site, and the first lecture and first assignment opened without a login on November 11, 2026
- •Has no exams: assignments end in an in-person oral checkoff and a research project is 50% of the grade
- •An NVIDIA infrastructure engineer is on the teaching staff, so you see that production stack directly, at the cost of a strong NVIDIA focus
- •The Spring 2026 site links student project blogs, including a study of self-summarizing reasoning trained with GRPO
Who It’s For
Best For
- ✓ML engineers moving from model use into training and serving at scale
- ✓Infrastructure engineers who need to reason about GPU clusters and inference economics
- ✓Graduate students looking for current research directions across pre-training, post-training, and inference
Prerequisites
- •No formal prerequisites are published; the lecture topics assume fluency with deep learning and PyTorch
- •Comfort with GPU and systems concepts such as memory bandwidth, interconnects, and parallel training
- •Your own multi-GPU compute if you want to attempt the assignments outside Berkeley
FAQ
What is Scalable AI: Bridging Theory, Understanding, and Practice (UC Berkeley EE 290 / 194, Fall 2026)?
Scalable AI is UC Berkeley's EE 290 / 194 course on building, training, and serving large language models, and it suits ML engineers and systems-minded students. After working through its slides and assignments, you can reason about GPU performance limits, plan pre-training and post-training runs, and compare production inference engines.
Is Scalable AI: Bridging Theory, Understanding, and Practice (UC Berkeley EE 290 / 194, Fall 2026) free?
Scalable AI: Bridging Theory, Understanding, and Practice (UC Berkeley EE 290 / 194, Fall 2026) is free to access.
What level is Scalable AI: Bridging Theory, Understanding, and Practice (UC Berkeley EE 290 / 194, Fall 2026) for?
Scalable AI: Bridging Theory, Understanding, and Practice (UC Berkeley EE 290 / 194, Fall 2026) is aimed at a advanced audience. Recommended background: No formal prerequisites are published; the lecture topics assume fluency with deep learning and PyTorch, Comfort with GPU and systems concepts such as memory bandwidth, interconnects, and parallel training, Your own multi-GPU compute if you want to attempt the assignments outside Berkeley.
How long does Scalable AI: Bridging Theory, Understanding, and Practice (UC Berkeley EE 290 / 194, Fall 2026) take?
Expect roughly One semester (Aug 26 to Dec 18, 2026): 28 class sessions, about 42 hours of class time, plus four assignments of about 3-4 weeks each. Most learners work through it at their own pace.
What will I learn from Scalable AI: Bridging Theory, Understanding, and Practice (UC Berkeley EE 290 / 194, Fall 2026)?
You'll learn: Model LLM performance limits with roofline analysis, memory bottlenecks, and token economics; Compare MoE, sparse, and long-context architectures under real hardware and serving constraints; Choose training parallelism strategies and study FlashAttention and SonicMoE in modern model stacks; Prepare pre-training data with deduplication, filtering, data mixtures, and NeMo Curator pipelines; Build supervised fine-tuning datasets with NeMo Data Designer and design RL environments; Run RL post-training with NeMo RL and NeMo Gym, following the Nemotron-NanoV3 case study; Compare Dynamo, TensorRT-LLM, vLLM, and SGLang for high-performance LLM inference in practice; Apply context engineering, harness design, deployment guardrails, and open-weight safety to LLM applications.
Topics
Sources
This page was written from 4 sources, 1 on domains other than scalable-ai.eecs.berkeley.edu.