Transformers in Practice
by DeepLearning.AI
Open the black box you call every day — from the autoregressive loop to KV cache, flash attention and speculative decoding.
Overview
Transformers in Practice is an eleven-hour DeepLearning.AI course produced with AMD and taught by Sharon Zhou, AMD's VP of Engineering & AI; it was announced in May 2026. It is organized as three modules, each closing with a graded lab and a 30-minute quiz. Module one, Observed Behavior, works outward from what you can already see at the API: the autoregressive loop, token sampling, how sampling shapes output, constrained generation with a finite-state machine for structured outputs, grounding in context, and thinking and reasoning — each paired with a visualization notebook. Module two, LLM Internals and Attention, opens the decoder-only architecture: attention, interpretable attention heads, positional encoding, model layers and decoding, including an hour-long notebook that decodes intermediate layers to show what the model has computed partway up the stack. Module three, Scaling and Deploying, is the production half — putting the model in production, quantization, KV cache, flash attention, speculative decoding, and the issues that arise in real deployments. Roughly a dozen lessons are runnable code examples rather than video, and the course opens with a conversation between Zhou and Andrew Ng. Its distinguishing angle is hardware: AMD's involvement means the efficiency material is framed around what happens on the GPU rather than as a vendor-neutral abstraction. Free to audit; graded labs and the certificate require DeepLearning.AI Pro.
At a Glance
- Topic
- Models
- Level
- Intermediate
- Format
- Course
- Cost
- Freemium
- Duration
- ~11 hours across 3 modules, self-paced
- Provider
- DeepLearning.AI
- Hands-on
- Yes — code/exercises
- Certificate
- Available
What You’ll Learn
- ✓Trace the autoregressive loop that produces model output one token at a time
- ✓Compare token sampling strategies and see how each one reshapes generated output
- ✓Implement constrained generation with a finite-state machine for structured outputs
- ✓Read attention, positional encoding and layer behavior inside a decoder-only transformer
- ✓Decode intermediate layers to inspect what a model computes partway through the stack
- ✓Apply quantization, KV caching and flash attention to cut inference cost on GPUs
- ✓Reason about speculative decoding and the GPU bottlenecks behind production latency
Highlights
- •Announced May 2026 and built with AMD, so the efficiency material is grounded in GPU behavior rather than left as vendor-neutral hand-waving
- •Eleven hours is unusually long for a DeepLearning.AI partner course — this is a full course, not a one-hour short course
- •About a dozen visualization notebooks, including hour-long labs on interpretable attention heads, intermediate-layer decoding and speculative decoding
- •Covers speculative decoding, flash attention and KV cache, which most transformer explainers stop short of
- •Taught by Sharon Zhou, AMD's VP of Engineering & AI, and opens with a conversation between her and Andrew Ng
Who It’s For
Best For
- ✓Engineers who call LLM APIs daily but have never opened the architecture
- ✓ML engineers responsible for inference cost, latency or GPU utilization
- ✓Developers debugging hallucinations, truncation or non-determinism in generation
- ✓Anyone about to do serving work with vLLM, SGLang or TensorRT-LLM
Prerequisites
- •Comfortable using LLMs through an API or chat interface
- •Basic neural network concepts such as weights, layers and training
- •Python, for the visualization and graded notebooks
FAQ
What is Transformers in Practice?
An eleven-hour DeepLearning.AI course built with AMD and taught by Sharon Zhou, AMD's VP of Engineering & AI, for developers who use LLMs through APIs but treat them as black boxes. Across three graded modules it moves from observable generation behavior, through attention and layer internals, to the inference optimizations — quantization, KV caching, flash attention, speculative decoding — that actually decide production latency and cost.
Is Transformers in Practice free?
Transformers in Practice offers free content, with paid options for certificates or premium features.
What level is Transformers in Practice for?
Transformers in Practice is aimed at a intermediate audience. Recommended background: Comfortable using LLMs through an API or chat interface, Basic neural network concepts such as weights, layers and training, Python, for the visualization and graded notebooks.
How long does Transformers in Practice take?
Expect roughly ~11 hours across 3 modules, self-paced. Most learners work through it at their own pace.
What will I learn from Transformers in Practice?
You'll learn: Trace the autoregressive loop that produces model output one token at a time; Compare token sampling strategies and see how each one reshapes generated output; Implement constrained generation with a finite-state machine for structured outputs; Read attention, positional encoding and layer behavior inside a decoder-only transformer; Decode intermediate layers to inspect what a model computes partway through the stack; Apply quantization, KV caching and flash attention to cut inference cost on GPUs; Reason about speculative decoding and the GPU bottlenecks behind production latency.
Topics
Sources
This page was written from 2 sources, 1 on domains other than deeplearning.ai.