MLModelsFrameworks

Machine Learning Systems: The AI Engineering Textbook (MLSysBook)

by Harvard University (Vijay Janapa Reddi)

IntermediateBookFreeSelf-paced; Volume I is 14 chapters plus 5 appendices, with TinyTorch's 20 build-it-yourself modules and Marimo labs as optional hands-on work

Harvard's free ML systems textbook, from data pipelines and training to compression, hardware acceleration, serving and MLOps.

Start LearningAdded Nov 18, 2026 · Updated Nov 18, 2026

Overview

Machine Learning Systems (MLSysBook) grew out of Harvard's CS249r course and is written by Vijay Janapa Reddi of Harvard University. The site describes four volumes: Volume I, Introduction to Machine Learning Systems, is released and listed as an MIT Press 2026 title; Volume II, Scaling Machine Learning Systems, is in preview for 2027; Volumes III (Agentic Machine Learning Systems) and IV (Physical AI) are open drafts in preparation. Volume I is organized into four parts. Foundations covers system paradigms, the ML lifecycle and data engineering. Development covers neural computation, network architectures, ML frameworks and model training. Optimization covers data selection, model compression, hardware acceleration and benchmarking. Deployment covers model serving, ML operations and responsible engineering. Appendices cover data, algorithm and machine foundations plus a glossary. The book ships with companion material: TinyTorch, an MIT-licensed ML framework you build from scratch in 20 modules; interactive Marimo notebook labs; hardware kits that deploy models to Arduino, Seeed, Grove and Raspberry Pi boards; the MLSys-im analytical performance simulator; StaffML interview drills; and 44 Beamer lecture decks with speaker notes. The Volume I page showed version v0.8.1, last updated November 7, 2026. The GitHub repo (harvard-edge/cs249r_book) had about 30.2k stars and 3.7k forks when checked on November 18, 2026, and the text is licensed CC BY-NC-ND 4.0.

At a Glance

Topic
ML
Level
Intermediate
Format
Book
Cost
Free
Duration
Self-paced; Volume I is 14 chapters plus 5 appendices, with TinyTorch's 20 build-it-yourself modules and Marimo labs as optional hands-on work
Provider
Harvard University (Vijay Janapa Reddi)
Hands-on
Yes — code/exercises
Certificate
None

What You’ll Learn

  • ✓How ML system paradigms and the ML lifecycle shape engineering decisions from data to deployment
  • ✓How data engineering pipelines are designed for training and serving machine learning models
  • ✓How ML frameworks implement neural computation, network architectures and the model training loop
  • ✓How data selection and model compression reduce training and inference cost
  • ✓How hardware acceleration and benchmarking methods determine real model performance on devices
  • ✓How model serving and ML operations keep models reliable once they are deployed in production
  • ✓How to build a small ML framework from scratch through the TinyTorch module sequence
  • ✓How to deploy trained models onto microcontroller and Raspberry Pi hardware with the companion kits

Highlights

  • •Free to read online, with Volume I listed as an MIT Press 2026 title
  • •Around 30.2k GitHub stars and an active repo with over 26,000 commits on the dev branch (checked November 18, 2026)
  • •Ships a full teaching kit: TinyTorch, Marimo labs, hardware kits, a performance simulator and 44 lecture decks
  • •Covers the whole stack from data engineering to MLOps, where most LLM courses stop at the model
  • •Volumes on scaling, agentic ML systems and physical AI are being written in the open

Who It’s For

Best For

  • ✓Software engineers moving into ML infrastructure or MLOps roles
  • ✓AI engineers who use models daily and want to understand training, compression and serving trade-offs
  • ✓Instructors building an ML systems course who want slides and labs ready to adopt
  • ✓Embedded and hardware engineers who want to run models on edge devices

Prerequisites

  • •Python programming proficiency with some NumPy familiarity
  • •Undergraduate linear algebra, calculus and probability
  • •Prior ML experience helps but is not required; the Neural Computation chapter supplies the background

FAQ

What is Machine Learning Systems: The AI Engineering Textbook (MLSysBook)?

Machine Learning Systems is a free online textbook by Harvard's Vijay Janapa Reddi for engineers who know some Python and ML and want to understand how models actually get trained, optimized, deployed and operated. After Volume I you can reason about compression, hardware acceleration, benchmarking, serving and MLOps trade-offs instead of treating them as black boxes.

Is Machine Learning Systems: The AI Engineering Textbook (MLSysBook) free?

Machine Learning Systems: The AI Engineering Textbook (MLSysBook) is free to access.

What level is Machine Learning Systems: The AI Engineering Textbook (MLSysBook) for?

Machine Learning Systems: The AI Engineering Textbook (MLSysBook) is aimed at a intermediate audience. Recommended background: Python programming proficiency with some NumPy familiarity, Undergraduate linear algebra, calculus and probability, Prior ML experience helps but is not required; the Neural Computation chapter supplies the background.

How long does Machine Learning Systems: The AI Engineering Textbook (MLSysBook) take?

Expect roughly Self-paced; Volume I is 14 chapters plus 5 appendices, with TinyTorch's 20 build-it-yourself modules and Marimo labs as optional hands-on work. Most learners work through it at their own pace.

What will I learn from Machine Learning Systems: The AI Engineering Textbook (MLSysBook)?

You'll learn: How ML system paradigms and the ML lifecycle shape engineering decisions from data to deployment; How data engineering pipelines are designed for training and serving machine learning models; How ML frameworks implement neural computation, network architectures and the model training loop; How data selection and model compression reduce training and inference cost; How hardware acceleration and benchmarking methods determine real model performance on devices; How model serving and ML operations keep models reliable once they are deployed in production; How to build a small ML framework from scratch through the TinyTorch module sequence; How to deploy trained models onto microcontroller and Raspberry Pi hardware with the companion kits.

Topics

ml systemsmlopsmodel compressionhardware accelerationtextbooktinyml

Sources

This page was written from 4 sources, 2 on domains other than mlsysbook.ai.

  1. 1.mlsysbook.ai — mlsysbook.aivendor
  2. 2.mlsysbook.ai — vol1vendor
  3. 3.github.com — cs249r book
  4. 4.hn.algolia.com — search