Neural Networks: Zero to Hero
by Andrej Karpathy
Build a GPT from scratch, line by line, with one of the field's best teachers.
Overview
This free lecture series is arguably the best way to truly understand how modern language models work. Karpathy (co-founder of OpenAI, ex-Tesla) starts from autograd (micrograd) and builds up through makemore to a full, from-scratch GPT — writing and explaining every line. By the end you understand backpropagation, attention, and transformer training at the level of the code itself. Demanding but transformative.
At a Glance
- Topic
- Models
- Level
- Advanced
- Format
- Video
- Cost
- Free
- Duration
- ~10+ hours (series)
- Provider
- Andrej Karpathy
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Backpropagation and autograd from scratch
- ✓Language modeling from bigrams to transformers
- ✓Building and training a GPT line by line
- ✓The internals of attention and transformer blocks
Highlights
- •Taught by Andrej Karpathy
- •Everything built from scratch in code
- •The definitive 'understand LLMs deeply' resource
Who It’s For
Best For
- ✓Engineers who want to understand LLMs at the code level
Prerequisites
- •Solid Python
- •Calculus basics
- •Some ML exposure
FAQ
What is Neural Networks: Zero to Hero?
Andrej Karpathy's acclaimed video series that builds neural networks and a GPT language model from first principles in code.
Is Neural Networks: Zero to Hero free?
Neural Networks: Zero to Hero is free to access.
What level is Neural Networks: Zero to Hero for?
Neural Networks: Zero to Hero is aimed at a advanced audience. Recommended background: Solid Python, Calculus basics, Some ML exposure.
How long does Neural Networks: Zero to Hero take?
Expect roughly ~10+ hours (series). Most learners work through it at their own pace.
What will I learn from Neural Networks: Zero to Hero?
You'll learn: Backpropagation and autograd from scratch; Language modeling from bigrams to transformers; Building and training a GPT line by line; The internals of attention and transformer blocks.