The Illustrated Transformer
by Jay Alammar
The illustrated explainer that made the transformer architecture click for a generation of engineers.
Overview
Written on 27 June 2018 by Jay Alammar — whose stated mission is 'visualizing machine learning one concept at a time' and who later co-authored the O'Reilly book Hands-On Large Language Models — this post is the reason a generation of engineers understand attention. It proceeds through fourteen short sections: A High-Level Look, Bringing The Tensors Into The Picture, Now We're Encoding!, Self-Attention at a High Level, Self-Attention in Detail, Matrix Calculation of Self-Attention, The Beast With Many Heads, Representing The Order of The Sequence Using Positional Encoding, The Residuals, The Decoder Side, The Final Linear and Softmax Layer, Recap Of Training, The Loss Function, and Go Forth And Transform. The method is to open one box at a time: a translation model becomes an encoder stack and a decoder stack, an encoder becomes self-attention plus a feed-forward network, and self-attention becomes the query, key and value projections with their scaled dot product — first as vectors, then as matrices. It is a reading, not a coding exercise: no notebooks, no exercises, no runnable code, though it links the original Attention Is All You Need paper, TensorFlow implementations and Harvard's annotated guide for the next step. Thirteen community translations exist, and the post keeps returning to the Hacker News front page, most recently in December 2025 at 500 points. Its one real limitation is age: it describes the 2017 encoder-decoder transformer, not the decoder-only stacks, rotary embeddings, grouped-query attention or KV caching that current LLMs are built on.
At a Glance
- Topic
- Models
- Level
- Intermediate
- Format
- Guide
- Cost
- Free
- Duration
- ~45 min read, no code to run
- Provider
- Jay Alammar
- Hands-on
- No
- Certificate
- None
What You’ll Learn
- ✓How self-attention scores a token against every other token in the sequence
- ✓What the query, key and value projections actually represent
- ✓The matrix form of self-attention and why it vectorizes so well
- ✓Why multi-head attention gives the model several representation subspaces
- ✓How positional encoding injects word order into an order-blind mechanism
- ✓Where residual connections and layer normalization sit inside each block
- ✓How the decoder attends to encoder output while generating tokens
- ✓What the final linear and softmax layers do to produce word probabilities
Highlights
- •The canonical visual explainer — thirteen community translations
- •Returned to the Hacker News front page at 500 points in December 2025, seven years after publication
- •Opens one abstraction level at a time; nothing is left as 'and then magic'
- •Free, no signup, roughly 45 minutes, readable on a phone
- •Links onward to the paper, the Annotated Transformer and reference implementations
Who It’s For
Best For
- ✓Engineers who need transformer intuition before reading the original paper
- ✓Anyone who has used an LLM API and wants to know what is inside
- ✓Interview preparation where you have to explain attention out loud
- ✓Instructors looking for diagrams that already work
Prerequisites
- •Basic neural network familiarity: layers, weights, softmax, backpropagation
- •Comfort with vectors and matrix multiplication
- •No Python needed — there is no code to run
FAQ
What is The Illustrated Transformer?
Jay Alammar's diagram-first walkthrough of the transformer architecture, published June 2018 and still the most-recommended conceptual explainer of the design underneath every modern LLM. It builds up from a black-box translation model to self-attention with queries, keys and values, its matrix form, multi-head attention, positional encoding, residuals, the decoder, and the final linear-plus-softmax layer.
Is The Illustrated Transformer free?
The Illustrated Transformer is free to access.
What level is The Illustrated Transformer for?
The Illustrated Transformer is aimed at a intermediate audience. Recommended background: Basic neural network familiarity: layers, weights, softmax, backpropagation, Comfort with vectors and matrix multiplication, No Python needed — there is no code to run.
How long does The Illustrated Transformer take?
Expect roughly ~45 min read, no code to run. Most learners work through it at their own pace.
What will I learn from The Illustrated Transformer?
You'll learn: How self-attention scores a token against every other token in the sequence; What the query, key and value projections actually represent; The matrix form of self-attention and why it vectorizes so well; Why multi-head attention gives the model several representation subspaces; How positional encoding injects word order into an order-blind mechanism; Where residual connections and layer normalization sit inside each block; How the decoder attends to encoder output while generating tokens; What the final linear and softmax layers do to produce word probabilities.
Topics
Sources
This page was written from 3 sources, 2 on domains other than jalammar.github.io.