MLModelsFine-Tuning

Machine Learning Engineering Open Book

by Stas Bekman

AdvancedBookFreeSelf-paced reference in 7 parts; read chapter by chapter as needed

A free, battle-tested field manual for training and serving LLMs on real GPU clusters

Start LearningAdded Sep 11, 2026 · Updated Sep 11, 2026

Overview

Machine Learning Engineering Open Book is a free, CC-BY-SA 4.0 licensed book maintained on GitHub by Stas Bekman, whose experience includes training BLOOM-176B (2022) and IDEFICS-80B (2023) and building RAG models at Contextual.AI (2024). It is a practitioner's reference for training and serving large language and multimodal models on real GPU clusters. The book is organised in seven parts: Insights (the AI battlefield, how to choose a cloud provider, when upgrading GPUs is worth it); Hardware (compute accelerators, storage, network); Orchestration (orchestration systems and SLURM); Training (model parallelism, performance, fault tolerance, reproducibility, instabilities, checkpoints, hyper-parameters and initialization, tensor precision and data types, emulating a multi-node setup on one node, re-training Hugging Face Hub models from scratch, datasets); Inference; Development (debugging, troubleshooting and testing); and Resources. The Inference part is a compact serving primer covering prefill versus decode, the memory anatomy of weights, KV cache and activations, static and continuous batching, paged attention, speculative decoding, guided generation, tensor and pipeline parallelism, TTFT and TPOT metrics, benchmarking, and a comparison of vLLM, SGLang, TensorRT-LLM, DeepSpeed-FastGen, llama.cpp, LMDeploy and MLC-LLM. It is actively maintained, with commits as recent as September 2026 (including a torch 2.14 update and new accelerator notes), and the repository has about 19,000 stars.

At a Glance

Topic: ML
Level: Advanced
Format: Book
Cost: Free
Duration: Self-paced reference in 7 parts; read chapter by chapter as needed
Provider: Stas Bekman
Hands-on: Yes — code/exercises
Certificate: None

What You’ll Learn

✓Size compute, storage and network requirements when planning a GPU cluster for LLM training
✓Decide which cloud provider fits and when upgrading to newer GPUs actually pays off
✓Run and schedule multi-node training jobs with SLURM and other orchestration systems
✓Apply model parallelism, precision choices and checkpointing to keep large training runs going
✓Diagnose training instabilities and reproducibility failures, and debug distributed PyTorch jobs
✓Explain prefill versus decode, KV cache memory, continuous batching and speculative decoding for inference
✓Measure serving performance with TTFT and TPOT and compare vLLM, SGLang and TensorRT-LLM

Highlights

•Written by an engineer who trained BLOOM-176B and IDEFICS-80B, giving rare first-hand large-cluster experience
•Actively maintained: commits as recent as September 2026, including a torch 2.14 update and new accelerator coverage
•Covers the unglamorous infrastructure courses usually skip: storage, networking, SLURM, fault tolerance and checkpointing
•Hacker News practitioners called it 'gold' for debugging LLM training setups (349-point discussion)
•Free and CC-BY-SA 4.0 licensed, readable directly on GitHub with no signup

Who It’s For

Best For

✓ML infrastructure engineers running multi-GPU or multi-node LLM training
✓Engineers self-hosting LLM inference who need to understand serving trade-offs
✓Tech leads choosing cloud providers and GPU generations
✓Researchers moving from single-GPU experiments to cluster-scale training

Prerequisites

•Solid Python and PyTorch experience
•Comfort with Linux, the shell and basic GPU concepts
•Understanding of transformer training fundamentals; this is not a beginner ML text

FAQ

What is Machine Learning Engineering Open Book?

Machine Learning Engineering Open Book is Stas Bekman's free, open-source book on the infrastructure side of large language model work, aimed at engineers who train or serve big models on GPU clusters. After reading it you can plan hardware, orchestrate multi-node training, keep long runs stable, and reason about inference performance and serving frameworks.

Is Machine Learning Engineering Open Book free?

Machine Learning Engineering Open Book is free to access.

What level is Machine Learning Engineering Open Book for?

Machine Learning Engineering Open Book is aimed at a advanced audience. Recommended background: Solid Python and PyTorch experience, Comfort with Linux, the shell and basic GPU concepts, Understanding of transformer training fundamentals; this is not a beginner ML text.

How long does Machine Learning Engineering Open Book take?

Expect roughly Self-paced reference in 7 parts; read chapter by chapter as needed. Most learners work through it at their own pace.

What will I learn from Machine Learning Engineering Open Book?

You'll learn: Size compute, storage and network requirements when planning a GPU cluster for LLM training; Decide which cloud provider fits and when upgrading to newer GPUs actually pays off; Run and schedule multi-node training jobs with SLURM and other orchestration systems; Apply model parallelism, precision choices and checkpointing to keep large training runs going; Diagnose training instabilities and reproducibility failures, and debug distributed PyTorch jobs; Explain prefill versus decode, KV cache memory, continuous batching and speculative decoding for inference; Measure serving performance with TTFT and TPOT and compare vLLM, SGLang and TensorRT-LLM.

Topics

ML engineeringLLM trainingGPU clustersLLM inferenceSLURMdistributed training

Sources

This page was written from 5 sources, 1 on domains other than github.com.