Dive into Deep Learning (D2L)
by Aston Zhang, Zachary C. Lipton, Mu Li and Alexander J. Smola
A free, executable deep learning textbook in four frameworks, used at 500 universities.
Overview
Dive into Deep Learning is a free, open-source textbook written as executable Jupyter notebooks, so every equation in the text has running code beneath it and every figure is regenerated from that code rather than drawn separately. It is written by Aston Zhang, Zachary C. Lipton, Mu Li and Alexander J. Smola, with additional volume-two chapters contributed by authors at institutions including UPenn, CMU and NYU, and is published by Cambridge University Press while remaining free online at version 1.0.3. Its distinguishing feature is multi-framework code: the same chapter is available implemented in PyTorch, NumPy/MXNet, JAX and TensorFlow, which makes it unusually useful for engineers who learned one framework and now need to read model code written in another. The 23 chapters run from preliminaries and linear neural networks through multilayer perceptrons and the builders' guide, then convolutional and recurrent architectures, attention mechanisms and transformers, computer vision and natural language processing applications, optimisation algorithms and computational performance, and finish with reinforcement learning, Gaussian processes, hyperparameter optimisation, generative adversarial networks, recommender systems and mathematics-for-deep-learning appendices. It has been adopted at 500 universities in 70 countries including Stanford, MIT, Harvard and Cambridge, and the GitHub repository has passed 29,000 stars, so errata are fixed in the open and the framework ports are maintained rather than abandoned after the first edition.
At a Glance
- Topic
- ML
- Level
- All Levels
- Format
- Book
- Cost
- Free
- Duration
- 23 chapters; a full pass is a semester of work, individual chapters ~3-5 hours
- Provider
- Aston Zhang, Zachary C. Lipton, Mu Li and Alexander J. Smola
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Implement linear regression and multilayer perceptron models from raw tensors upward
- ✓Build convolutional architectures and understand why they generalise on images
- ✓Implement attention mechanisms and a transformer from scratch with running code
- ✓Apply optimisation algorithms and diagnose why a training run stalls
- ✓Read and write the same model in PyTorch, JAX and TensorFlow
- ✓Work through computer vision and NLP application chapters end to end
- ✓Work through the appendices on linear algebra, calculus and probability
Highlights
- •Every chapter is an executable Jupyter notebook, so the text and the code cannot drift apart
- •The same material in PyTorch, JAX, TensorFlow and NumPy/MXNet, which no other deep learning textbook offers
- •Adopted at 500 universities across 70 countries including Stanford, MIT, Harvard and Cambridge
- •Free online in full while also being a Cambridge University Press book, with no paywalled chapters
- •23 chapters take you from tensors to transformers, GANs and reinforcement learning in one consistent notation
Who It’s For
Best For
- ✓Engineers who want depth rather than another framework API tutorial
- ✓Students taking a deep learning course that assigns it as a text
- ✓Practitioners who need to read model code across framework boundaries
- ✓Self-taught developers filling in the mathematics they skipped
Prerequisites
- •Python programming, including NumPy-style array manipulation
- •Undergraduate linear algebra, calculus and probability, with appendices covering gaps
- •No prior deep learning experience needed, since the book starts from first principles
FAQ
What is Dive into Deep Learning (D2L)?
A complete deep learning textbook that is also a set of runnable notebooks, free online and published by Cambridge University Press. Its 23 chapters run from tensors and linear models through convolutional networks, attention, transformers, optimisation, GANs and reinforcement learning, with code listings available in PyTorch, JAX, TensorFlow and NumPy/MXNet. Adopted at 500 universities across 70 countries including Stanford, MIT, Harvard and Cambridge.
Is Dive into Deep Learning (D2L) free?
Dive into Deep Learning (D2L) is free to access.
What level is Dive into Deep Learning (D2L) for?
Dive into Deep Learning (D2L) is aimed at a all levels audience. Recommended background: Python programming, including NumPy-style array manipulation, Undergraduate linear algebra, calculus and probability, with appendices covering gaps, No prior deep learning experience needed, since the book starts from first principles.
How long does Dive into Deep Learning (D2L) take?
Expect roughly 23 chapters; a full pass is a semester of work, individual chapters ~3-5 hours. Most learners work through it at their own pace.
What will I learn from Dive into Deep Learning (D2L)?
You'll learn: Implement linear regression and multilayer perceptron models from raw tensors upward; Build convolutional architectures and understand why they generalise on images; Implement attention mechanisms and a transformer from scratch with running code; Apply optimisation algorithms and diagnose why a training run stalls; Read and write the same model in PyTorch, JAX and TensorFlow; Work through computer vision and NLP application chapters end to end; Work through the appendices on linear algebra, calculus and probability.
Topics
Sources
This page was written from 3 sources, 2 on domains other than d2l.ai.