GPU Glossary
by Modal Labs
Finally understand the GPU stack — SMs, warps, CUDA cores, drivers, and what actually makes inference slow.
Overview
Modal Labs built the GPU Glossary to fix the fragmentation of GPU documentation, where concepts like Streaming Multiprocessor architecture, compute capability, and compiler flags live in separate manuals and are hard to relate to each other. It is a hypertext document whose pages are all inter-linked, so you can jump from a warp scheduler to the CUDA programming model's notion of threads and back. The glossary is organized into four main sections: Device Hardware (streaming multiprocessors, CUDA cores, memory architecture, schedulers), Device Software (the CUDA programming model, threads, warps, kernels, memory hierarchy), Host Software (drivers, APIs, compilers, and libraries such as cuBLAS and cuDNN, plus profiling tools), and Performance (bottlenecks, occupancy, memory bandwidth, and optimization metrics), alongside contributor and README pages. It is freely readable online, sourced from a public GitHub repository with a feedback and contribution path, and is aimed at developers and engineers who need to reason about hardware, software, and performance as one connected system rather than memorize isolated terms.
At a Glance
- Topic
- ML
- Level
- Intermediate
- Format
- Guide
- Cost
- Free
- Duration
- ~2-3 hours to read through; usable as an ongoing reference
- Provider
- Modal Labs
- Hands-on
- No
- Certificate
- None
What You’ll Learn
- ✓How GPU hardware is organized: streaming multiprocessors, CUDA cores, schedulers, and the memory hierarchy
- ✓The CUDA programming model in practice — threads, warps, blocks, and kernels
- ✓What the host-side stack does: drivers, runtime APIs, nvcc, cuBLAS/cuDNN, and profilers
- ✓How to reason about occupancy, memory bandwidth, and compute-vs-memory bound bottlenecks
- ✓Why the same model can be fast or slow depending on which layer of the stack limits it
Highlights
- •Every term is hyperlinked to related terms, so concepts connect across hardware, software, and performance
- •Written for practitioners by an inference-platform team, not a textbook
- •Open on GitHub and free to read — no signup required
- •Short, self-contained pages make it usable as a lookup reference mid-debug
Who It’s For
Best For
- ✓AI engineers optimizing LLM inference or training throughput
- ✓Developers new to CUDA who need the mental model before the API
- ✓Anyone sizing GPUs or debugging why a model is slower than expected
Prerequisites
- •Basic programming experience
- •Some exposure to deep learning training or inference
- •No CUDA experience required
FAQ
What is GPU Glossary?
The GPU Glossary is Modal Labs' hyperlinked reference that explains GPU computing end to end, from streaming multiprocessors up through drivers, compilers, and performance metrics. It is for AI engineers who train or serve models and keep hitting GPU terminology they can't connect across layers of the stack.
Is GPU Glossary free?
GPU Glossary is free to access.
What level is GPU Glossary for?
GPU Glossary is aimed at a intermediate audience. Recommended background: Basic programming experience, Some exposure to deep learning training or inference, No CUDA experience required.
How long does GPU Glossary take?
Expect roughly ~2-3 hours to read through; usable as an ongoing reference. Most learners work through it at their own pace.
What will I learn from GPU Glossary?
You'll learn: How GPU hardware is organized: streaming multiprocessors, CUDA cores, schedulers, and the memory hierarchy; The CUDA programming model in practice — threads, warps, blocks, and kernels; What the host-side stack does: drivers, runtime APIs, nvcc, cuBLAS/cuDNN, and profilers; How to reason about occupancy, memory bandwidth, and compute-vs-memory bound bottlenecks; Why the same model can be fast or slow depending on which layer of the stack limits it.