nanochat: The best ChatGPT that $100 can buy
by Andrej Karpathy
Train your own ChatGPT-style model end-to-end from a single, hackable ~8,000-line codebase.
Overview
nanochat is Andrej Karpathy's minimal, end-to-end codebase for building a small ChatGPT clone from scratch. Unlike his earlier nanoGPT (which only covered pre-training), nanochat implements the full loop in roughly 8,000 lines of clean, readable, minimal-dependency code: tokenization, base pre-training, mid-training on chat / multiple-choice / tool-use data, supervised fine-tuning (SFT), optional reinforcement learning, evaluation, and serving via a CLI and a ChatGPT-like web UI. A single 'speedrun' script runs the entire pipeline; a full run lands near $100 of compute in about four hours on an 8xH100 node, while a smaller GPT-2-grade run is faster and cheaper. Released in October 2025 under the MIT license, it is designed to be a hackable, maximally-forkable strong-baseline for learning how modern LLMs are actually built and served.
At a Glance
- Topic
- ML
- Level
- Advanced
- Format
- Interactive
- Cost
- Free
- Duration
- ~4-hour speedrun on an 8xH100 node (~$100)
- Provider
- Andrej Karpathy
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓The complete LLM lifecycle: tokenization, pre-training, mid-training, SFT, RL, eval, and serving
- ✓How to train a small ChatGPT-style model end-to-end on a single GPU node
- ✓How mid-training on chat, multiple-choice, and tool-use data shapes a base model into an assistant
- ✓How supervised fine-tuning and optional RL improve instruction-following
- ✓How to serve a trained model through a CLI and a web chat interface
Highlights
- •Full-stack pipeline in a single ~8,000-line, minimal-dependency repo
- •One 'speedrun' script runs tokenization through serving end-to-end
- •Maintained by Andrej Karpathy; actively developed with a public run leaderboard
- •MIT-licensed, maximally forkable and hackable for experimentation
Who It’s For
Best For
- ✓ML engineers who want to understand LLMs by building one from scratch
- ✓Practitioners studying the pre-training to fine-tuning to serving pipeline
- ✓Educators and self-learners who want a hackable strong-baseline codebase
Prerequisites
- •Strong Python and PyTorch skills
- •Working knowledge of transformers and neural-network training
- •Access to GPUs (an 8xH100 node for the full speedrun; smaller runs possible)
FAQ
What is nanochat: The best ChatGPT that $100 can buy?
A minimal, from-scratch, full-stack ChatGPT training and inference pipeline in a single dependency-light repository by Andrej Karpathy. Built for engineers who want to understand the entire modern LLM lifecycle by running and modifying every stage themselves, not just calling an API.
Is nanochat: The best ChatGPT that $100 can buy free?
nanochat: The best ChatGPT that $100 can buy is free to access.
What level is nanochat: The best ChatGPT that $100 can buy for?
nanochat: The best ChatGPT that $100 can buy is aimed at a advanced audience. Recommended background: Strong Python and PyTorch skills, Working knowledge of transformers and neural-network training, Access to GPUs (an 8xH100 node for the full speedrun; smaller runs possible).
How long does nanochat: The best ChatGPT that $100 can buy take?
Expect roughly ~4-hour speedrun on an 8xH100 node (~$100). Most learners work through it at their own pace.
What will I learn from nanochat: The best ChatGPT that $100 can buy?
You'll learn: The complete LLM lifecycle: tokenization, pre-training, mid-training, SFT, RL, eval, and serving; How to train a small ChatGPT-style model end-to-end on a single GPU node; How mid-training on chat, multiple-choice, and tool-use data shapes a base model into an assistant; How supervised fine-tuning and optional RL improve instruction-following; How to serve a trained model through a CLI and a web chat interface.