AI Systems Performance Engineering
by Chris Fregly (O'Reilly Media)
Tune CUDA kernels, PyTorch, multi-node training and disaggregated LLM inference on NVIDIA GPUs, with baseline-versus-optimized code for every chapter.
Overview
AI Systems Performance Engineering by Chris Fregly was published by O'Reilly Media in late 2025 (Ingram lists the paperback at 1,060 pages on December 16, 2025, at $99.99). Its 20 chapters move up the stack. Chapters 1 to 5 cover AI system hardware, OS, Docker and Kubernetes tuning, distributed networking communication and GPU-based storage I/O. Chapters 6 to 12 are a CUDA block on GPU architecture and occupancy, memory access patterns, warp efficiency and instruction-level parallelism, arithmetic intensity, intra-kernel pipelining with thread block clusters, CUDA streams and device-side kernel orchestration. Chapters 13 and 14 cover profiling and scaling PyTorch and the PyTorch compiler, XLA and OpenAI Triton backends. Chapters 15 to 19 cover inference at scale: multi-node parallelism and routing, profiling inference, disaggregated prefill and decode, KV cache tuning and adaptive inference engine optimizations. Chapter 20 covers AI-assisted performance optimization. The free GitHub repository (about 2.1k stars, 3,000+ commits, Apache-2.0) pairs a baseline and an optimized version of each workload in ch01 to ch20, adds labs on FlashAttention 4, persistent decode, KV optimization, MoE and block-scaled GEMM, and drives them through a Typer CLI with Nsight Systems and Nsight Compute profiles. The code targets NVIDIA Blackwell on a pinned PyTorch 2.9.1 with CUDA 13, and the repo also carries a 200+ item performance checklist and monthly meetup recordings through August 2026.
At a Glance
- Topic
- ML
- Level
- Advanced
- Format
- Book
- Cost
- Freemium
- Duration
- 1,060-page book in 20 chapters plus per-chapter GPU benchmarks and labs, self-paced
- Provider
- Chris Fregly (O'Reilly Media)
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Tune operating system, Docker and Kubernetes settings that bottleneck GPU training and inference nodes
- ✓Optimize multi-node networking communication and GPU-direct storage I/O for large training jobs
- ✓Profile and fix CUDA kernel memory access patterns, occupancy, warp efficiency and arithmetic intensity
- ✓Overlap work with intra-kernel pipelining, thread block clusters, CUDA streams and device-side kernel launches
- ✓Profile and scale PyTorch, and compare the PyTorch compiler, XLA and OpenAI Triton backends
- ✓Design multi-node inference parallelism and request routing for large language model serving
- ✓Run disaggregated prefill and decode and tune the KV cache for higher inference throughput
- ✓Benchmark baseline against optimized workloads with Nsight Systems and Nsight Compute through the repo CLI
Highlights
- •Covers the whole stack in one book, from Kubernetes and NCCL-level networking down to CUDA warps and up to disaggregated LLM serving
- •Every chapter has paired baseline and optimized benchmarks with correctness checks, so you measure the speedup yourself
- •Active Apache-2.0 repo (about 2.1k stars and 285 forks) with labs on FlashAttention 4, persistent decode and KV cache optimization, plus BOOK-ERRATA.md
- •Recommended in a May 2026 Hacker News thread on CUDA books as a title that 'might deserve a mention, even though it's not strictly CUDA'
Who It’s For
Best For
- ✓Inference engineers tuning vLLM-style serving, KV cache and prefill-decode disaggregation
- ✓ML infrastructure engineers running multi-node training on NVIDIA GPU clusters
- ✓PyTorch developers ready to move from torch.compile into custom CUDA and Triton kernels
- ✓Platform teams trying to cut GPU cost per token or per training step
Prerequisites
- •Working PyTorch and Python, and enough C++ to read CUDA kernels
- •Access to bare-metal NVIDIA GPUs for the labs; the code targets Blackwell (B200) with PyTorch 2.9.1 and CUDA 13, and virtualized runs are labeled non-canonical
- •Basic Linux and Kubernetes administration for the OS, networking and cluster chapters
FAQ
What is AI Systems Performance Engineering?
AI Systems Performance Engineering is Chris Fregly's O'Reilly book on making AI training and inference faster and cheaper on NVIDIA GPU clusters. It is for ML infrastructure and inference engineers who already write PyTorch. Afterwards you can profile and tune CUDA kernels, PyTorch compilation, distributed communication and prefill-decode serving, using the free Apache-2.0 code repository.
Is AI Systems Performance Engineering free?
AI Systems Performance Engineering offers free content, with paid options for certificates or premium features.
What level is AI Systems Performance Engineering for?
AI Systems Performance Engineering is aimed at a advanced audience. Recommended background: Working PyTorch and Python, and enough C++ to read CUDA kernels, Access to bare-metal NVIDIA GPUs for the labs; the code targets Blackwell (B200) with PyTorch 2.9.1 and CUDA 13, and virtualized runs are labeled non-canonical, Basic Linux and Kubernetes administration for the OS, networking and cluster chapters.
How long does AI Systems Performance Engineering take?
Expect roughly 1,060-page book in 20 chapters plus per-chapter GPU benchmarks and labs, self-paced. Most learners work through it at their own pace.
What will I learn from AI Systems Performance Engineering?
You'll learn: Tune operating system, Docker and Kubernetes settings that bottleneck GPU training and inference nodes; Optimize multi-node networking communication and GPU-direct storage I/O for large training jobs; Profile and fix CUDA kernel memory access patterns, occupancy, warp efficiency and arithmetic intensity; Overlap work with intra-kernel pipelining, thread block clusters, CUDA streams and device-side kernel launches; Profile and scale PyTorch, and compare the PyTorch compiler, XLA and OpenAI Triton backends; Design multi-node inference parallelism and request routing for large language model serving; Run disaggregated prefill and decode and tune the KV cache for higher inference throughput; Benchmark baseline against optimized workloads with Nsight Systems and Nsight Compute through the repo CLI.
Topics
Sources
This page was written from 4 sources, 2 on domains other than github.com.