CS329A: Self-Improving AI Agents
by Stanford University
Stanford's graduate reading map for agents that improve themselves
Overview
CS329A ran in Autumn 2025 on a Monday/Friday schedule in Skilling Auditorium and is listed in the Stanford Bulletin as a 3-unit graduate seminar with CS224N or CS229S as the prerequisite, plus Python fluency and hands-on experience calling LLM APIs. The twenty published sessions move from test-time compute scaling and robust verification, through learning from feedback with tools and code (ReAct, RLEF, Constitutional AI), multi-step reasoning and planning, and train-time scaling with RL (STaR, DeepSeekMath, DAPO), into open-ended evolution of self-improving agents, search and deep-research agents, agentic frameworks for software engineering, agent memory (MemGPT, Cartridges, CacheBlend) and agentic evaluation on long-horizon tasks. Each session links its required papers directly, which is the resource's real value to an outside reader. Assessment for enrolled students is three homeworks (50%), a project proposal, a midterm presentation and report, a 35% final research project and a poster session — so the course is oriented toward original research rather than framework tutorials. Guest lecturers include Denny Zhou, Thang Luong and Melvin Johnson of Google DeepMind, Misha Laskin of Reflection AI, Junchen Jiang of LMCache and the University of Chicago, and Danny Driess of Physical Intelligence. Chowdhery co-led PaLM; Mirhoseini worked on AlphaChip and runs Stanford's Scaling Intelligence Lab.
At a Glance
- Topic
- Agentic
- Level
- Advanced
- Format
- Course
- Cost
- Free
- Duration
- 20 sessions over ~11 weeks; budget 2-4 hours a week of paper reading to self-study the syllabus
- Provider
- Stanford University
- Hands-on
- No
- Certificate
- None
What You’ll Learn
- ✓Scale test-time compute with search, sampling and verification to improve model answers
- ✓Build robust verifiers and use them to filter and rank candidate agent trajectories
- ✓Apply RL post-training methods including STaR, DeepSeekMath and DAPO to reasoning models
- ✓Design multi-step reasoning and planning loops for long-horizon agentic workflows
- ✓Give agents durable memory using MemGPT, Cartridges and CacheBlend-style KV cache techniques
- ✓Evaluate long-horizon agents and understand where agentic benchmarks mislead you
- ✓Follow open-ended evolution research where agents rewrite their own scaffolds and prompts
- ✓Frame an original research project on self-improving agents from proposal to final report
Highlights
- •A curated per-session paper list — ReAct, Constitutional AI, STaR, DAPO, MemGPT, Search-o1, AlphaCode — instead of a survey lecture
- •Taught by Aakanksha Chowdhery (PaLM co-lead) and Azalia Mirhoseini (AlphaChip, Stanford Scaling Intelligence Lab)
- •Guest lectures from Denny Zhou, Thang Luong and Melvin Johnson (Google DeepMind), Misha Laskin (Reflection AI) and Danny Driess (Physical Intelligence)
- •One of the few graduate courses organised around self-improvement itself rather than agent frameworks or prompt patterns
- •Officially listed in the Stanford Bulletin at 3 units with a stated CS224N/CS229S prerequisite, so the difficulty claim is verifiable
Who It’s For
Best For
- ✓Researchers and PhD students working on reasoning, RL post-training or agent evaluation
- ✓Senior AI engineers who want the primary-source reading list behind self-improving agents
- ✓Teams building verifiers, reward models or long-horizon evaluation harnesses
- ✓Anyone who prefers reading the original papers over a framework walkthrough
Prerequisites
- •CS224N or CS229S level background in deep learning and NLP
- •Python fluency and hands-on experience calling large language model APIs
- •Comfort reading current arXiv research papers without a lecture to explain them — no recordings are published
FAQ
What is CS329A: Self-Improving AI Agents?
A Stanford graduate seminar on AI agents that continuously improve through interaction with themselves and their environment, taught by Aakanksha Chowdhery and Azalia Mirhoseini. It is aimed at researchers and senior engineers working on post-training, verification and long-horizon agents. The public site publishes the full session-by-session schedule with linked papers — no slides or lecture recordings — so treat it as a curated research reading map rather than a self-paced video course.
Is CS329A: Self-Improving AI Agents free?
CS329A: Self-Improving AI Agents is free to access.
What level is CS329A: Self-Improving AI Agents for?
CS329A: Self-Improving AI Agents is aimed at a advanced audience. Recommended background: CS224N or CS229S level background in deep learning and NLP, Python fluency and hands-on experience calling large language model APIs, Comfort reading current arXiv research papers without a lecture to explain them — no recordings are published.
How long does CS329A: Self-Improving AI Agents take?
Expect roughly 20 sessions over ~11 weeks; budget 2-4 hours a week of paper reading to self-study the syllabus. Most learners work through it at their own pace.
What will I learn from CS329A: Self-Improving AI Agents?
You'll learn: Scale test-time compute with search, sampling and verification to improve model answers; Build robust verifiers and use them to filter and rank candidate agent trajectories; Apply RL post-training methods including STaR, DeepSeekMath and DAPO to reasoning models; Design multi-step reasoning and planning loops for long-horizon agentic workflows; Give agents durable memory using MemGPT, Cartridges and CacheBlend-style KV cache techniques; Evaluate long-horizon agents and understand where agentic benchmarks mislead you; Follow open-ended evolution research where agents rewrite their own scaffolds and prompts; Frame an original research project on self-improving agents from proposal to final report.
Topics
Sources
This page was written from 3 sources, 2 on domains other than cs329a.stanford.edu.