GPU Glossary
by Modal
Decode GPU jargon — from Streaming Multiprocessor to roofline model — in one linked reference.
Overview
The GPU Glossary is a free, hyperlinked reference to NVIDIA GPU internals published by Modal Labs, the serverless GPU compute company, and released open source in February 2025 under CC BY 4.0 with the repository tooling under MIT. It is organised into four content sections plus a contributors page: Device Hardware (roughly eighteen entries covering Streaming Multiprocessors, CUDA Cores, Tensor Cores, register files, shared memory and the L1/L2 cache hierarchy), Device Software (about fifteen entries on the CUDA programming model — threads, warps, thread blocks, kernels, PTX and SASS), Host Software (about twenty-four entries on the CUDA Runtime and Driver APIs, nvcc, nvidia-smi, NVML, cuBLAS, cuDNN and the profiling tools), and Performance (about twenty-one entries added in September 2025 covering roofline models, occupancy, memory coalescing, arithmetic intensity and utilisation metrics). Every entry is a short page that links laterally to related terms, so it can be read linearly with the navigation arrows or explored non-linearly; the authors explicitly describe it as a hypertext document rather than a PDF or a textbook. Its stated purpose is to fix the fragmentation of NVIDIA's own documentation, where connecting Streaming Multiprocessor architecture to compute capability to an nvcc compiler flag currently means reading three unrelated manuals. It reached the Hacker News front page in January 2025 with 230 points and 70 comments, and the GitHub repository has over 670 stars plus a community Simplified Chinese translation.
At a Glance
- Topic
- ML
- Level
- Intermediate
- Format
- Documentation
- Cost
- Free
- Duration
- ~4-6 hours to read end to end; ~80 short linked entries, self-paced reference
- Provider
- Modal
- Hands-on
- No
- Certificate
- None
What You’ll Learn
- ✓Map NVIDIA hardware units — SMs, CUDA cores, Tensor Cores — onto the CUDA execution model
- ✓Explain how threads, warps, thread blocks and grids are scheduled onto Streaming Multiprocessors
- ✓Navigate the CUDA software stack from nvcc and PTX down to SASS machine code
- ✓Distinguish the CUDA Runtime API from the Driver API and know when each one is used
- ✓Read a roofline model and decide whether a kernel is compute-bound or memory-bound
- ✓Interpret occupancy, arithmetic intensity and memory coalescing when profiling GPU kernels
- ✓Identify where cuBLAS, cuDNN and the profiling tools sit in the host software stack
Highlights
- •Open sourced under CC BY 4.0 in February 2025; 670+ GitHub stars and a community Chinese translation
- •Hypertext design: every term links to the terms it depends on, so you drill down instead of searching
- •The Performance section, added September 2025, is unusually concrete about roofline, occupancy and utilisation metrics
- •Written by a GPU-infrastructure company, so terminology matches what actually appears in nvidia-smi and profiler output
- •Hacker News reviewers flagged the real gaps honestly: no cross-generation spec tables, and no single-page or PDF export
Who It’s For
Best For
- ✓Software engineers moving into GPU or inference-infrastructure work
- ✓ML engineers who need to read profiler output and CUDA error messages
- ✓Anyone reading CUDA kernel code or FlashAttention-style writeups for the first time
Prerequisites
- •Comfort with systems concepts: memory hierarchy, caches, threads and compilation
- •No CUDA experience required, though C/C++ familiarity helps in the Host Software section
FAQ
What is GPU Glossary?
A free, open-source hypertext reference to how NVIDIA GPUs actually work, published by Modal. Roughly eighty short linked entries span device hardware, the CUDA programming model, the host software stack and performance analysis. Read it end to end, or use it to decode the term that just appeared in your profiler output, a CUDA error message, or a kernel-optimisation blog post.
Is GPU Glossary free?
GPU Glossary is free to access.
What level is GPU Glossary for?
GPU Glossary is aimed at a intermediate audience. Recommended background: Comfort with systems concepts: memory hierarchy, caches, threads and compilation, No CUDA experience required, though C/C++ familiarity helps in the Host Software section.
How long does GPU Glossary take?
Expect roughly ~4-6 hours to read end to end; ~80 short linked entries, self-paced reference. Most learners work through it at their own pace.
What will I learn from GPU Glossary?
You'll learn: Map NVIDIA hardware units — SMs, CUDA cores, Tensor Cores — onto the CUDA execution model; Explain how threads, warps, thread blocks and grids are scheduled onto Streaming Multiprocessors; Navigate the CUDA software stack from nvcc and PTX down to SASS machine code; Distinguish the CUDA Runtime API from the Driver API and know when each one is used; Read a roofline model and decide whether a kernel is compute-bound or memory-bound; Interpret occupancy, arithmetic intensity and memory coalescing when profiling GPU kernels; Identify where cuBLAS, cuDNN and the profiling tools sit in the host software stack.
Topics
Sources
This page was written from 5 sources, 2 on domains other than modal.com.