Triton Documentation — Writing GPU Kernels in Python
by Triton (triton-lang)
Ten worked kernels — from vector add to fused attention — that teach GPU programming at block level in Python instead of CUDA.
Overview
Triton is a language and compiler for parallel programming that provides a Python-based environment for writing custom deep-learning compute kernels, created by Philippe Tillet and developed in the open at triton-lang/triton. Its design point is to raise the abstraction above CUDA: you express work in terms of blocks and tensors rather than individual threads, and the compiler handles memory coalescing, shared-memory allocation and tensor-core scheduling. The documentation is organised as Getting Started (Installation, Tutorials); a Python API reference covering triton, triton.language, triton.testing, Triton Semantics and triton.language.extra.cuda; Gluon, a newer lower-level programming model with its own overview, tutorials, examples and API reference; Triton MLIR Dialects documenting the dialects and ops the compiler lowers through; and a Programming Guide with an Introduction, Related Work, Debugging Triton and a Floating-Point Sanitizer (FpSan) page. The tutorial sequence is meant to be taken in order and runs Vector Addition, Fused Softmax, Matrix Multiplication, Low-Memory Dropout, Layer Normalization, Fused Attention, Libdevice function usage, Group GEMM, Persistent Matmul and Block Scaled Matrix Multiplication — a path that ends at the kernels real inference engines actually ship. The compiler backend is built on MLIR and targets NVIDIA GPUs at compute capability 8.0 and above, AMD GPUs on ROCm 6.2 and above, with CPU support in development. The repository has 20k stars across 6,740 commits.
At a Glance
- Topic
- ML
- Level
- Advanced
- Format
- Documentation
- Cost
- Free
- Duration
- ~6-10 hours to work through the ten tutorials; weeks of practice to write kernels that beat a vendor library
- Provider
- Triton (triton-lang)
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Write a Triton kernel and launch it with a grid, program IDs and block pointers
- ✓Reason about memory coalescing and shared memory at block rather than thread granularity
- ✓Fuse elementwise chains into one kernel to eliminate intermediate memory traffic
- ✓Implement fused softmax, layer normalization and fused attention from first principles
- ✓Autotune block sizes and num_warps, and benchmark against cuBLAS with triton.testing
- ✓Write high-performance matmul variants: grouped GEMM, persistent matmul, block-scaled matmul
- ✓Debug numerically wrong kernels using the debugging guide and the floating-point sanitizer
- ✓Read the MLIR dialects to understand what the compiler did with your kernel
Highlights
- •The primary source for the language behind torch.compile's generated kernels and much of vLLM/SGLang's custom ops
- •Tutorials are runnable, benchmarked kernels — each one plots throughput against a vendor baseline
- •The fused-attention tutorial is a real FlashAttention-style implementation, not a simplified teaching version
- •Genuinely portable: the same kernel source targets NVIDIA and AMD, with CPU support in progress
- •Documents Gluon, the lower-level model for when Triton's abstractions cost you performance
Who It’s For
Best For
- ✓Inference engineers chasing latency that framework-level optimisation cannot reach
- ✓ML systems engineers implementing operations no vendor library provides
- ✓Researchers prototyping novel attention or quantization schemes that need custom kernels
Prerequisites
- •Strong Python and PyTorch, including tensor layouts and strides
- •GPU fundamentals: memory hierarchy, warps/wavefronts, occupancy, arithmetic intensity
- •Access to an NVIDIA GPU at compute capability 8.0+ or an AMD GPU on ROCm 6.2+
FAQ
What is Triton Documentation — Writing GPU Kernels in Python?
The official documentation for Triton, the Python-embedded language and compiler for writing custom GPU kernels that power torch.compile and much of the modern inference stack. Written for engineers who have profiled a model, found a bottleneck no vendor library covers, and need to write the kernel themselves. After working through it you can write, autotune and debug block-level kernels for NVIDIA and AMD GPUs, and read the fused-attention implementations that production inference engines ship.
Is Triton Documentation — Writing GPU Kernels in Python free?
Triton Documentation — Writing GPU Kernels in Python is free to access.
What level is Triton Documentation — Writing GPU Kernels in Python for?
Triton Documentation — Writing GPU Kernels in Python is aimed at a advanced audience. Recommended background: Strong Python and PyTorch, including tensor layouts and strides, GPU fundamentals: memory hierarchy, warps/wavefronts, occupancy, arithmetic intensity, Access to an NVIDIA GPU at compute capability 8.0+ or an AMD GPU on ROCm 6.2+.
How long does Triton Documentation — Writing GPU Kernels in Python take?
Expect roughly ~6-10 hours to work through the ten tutorials; weeks of practice to write kernels that beat a vendor library. Most learners work through it at their own pace.
What will I learn from Triton Documentation — Writing GPU Kernels in Python?
You'll learn: Write a Triton kernel and launch it with a grid, program IDs and block pointers; Reason about memory coalescing and shared memory at block rather than thread granularity; Fuse elementwise chains into one kernel to eliminate intermediate memory traffic; Implement fused softmax, layer normalization and fused attention from first principles; Autotune block sizes and num_warps, and benchmark against cuBLAS with triton.testing; Write high-performance matmul variants: grouped GEMM, persistent matmul, block-scaled matmul; Debug numerically wrong kernels using the debugging guide and the floating-point sanitizer; Read the MLIR dialects to understand what the compiler did with your kernel.
Topics
Sources
This page was written from 3 sources, 1 on domains other than triton-lang.org.
- 1.triton-lang.org — indexvendor
- 2.triton-lang.org — indexvendor
- 3.github.com — triton