AI Engineering: Building Applications with Foundation Models
by Chip Huyen (O'Reilly Media)
The desk reference for building on foundation models, with two full chapters on evaluation.
Overview
AI Engineering: Building Applications with Foundation Models is Chip Huyen's O'Reilly book, published in 2025 and running roughly 500 pages across ten chapters. Huyen built NeMo, NVIDIA's generative AI framework, researched at Netflix, co-founded Claypot AI and taught machine learning at Stanford; her earlier O'Reilly title, Designing Machine Learning Systems, is a standard reference for the pre-foundation-model version of the same job. The premise here is that the workflow inverted: you now start from a product idea and an available model, and consider training something custom later, if ever. The chapters follow that order. Chapter one frames the AI engineering stack against ML engineering and full-stack work. Chapter two covers foundation model internals, post-training and sampling, including structured outputs. Chapters three and four are an unusually long treatment of evaluation — perplexity and cross-entropy, functional correctness, AI-as-judge and its limits, comparative ranking, then model selection and how to build an evaluation pipeline. Chapter five is prompt engineering including defenses against injection and prompt extraction. Chapter six covers RAG retrieval algorithms alongside agents, tools, planning and agent failure modes. Chapters seven and eight cover finetuning, memory math, quantization and PEFT, then dataset curation and synthesis. Chapter nine covers inference optimization, and chapter ten covers production architecture — guardrails, model routers, caches, observability — plus user feedback design. The companion repository, chiphuyen/aie-book, carries chapter summaries and a curated resource list.
At a Glance
- Topic
- ML
- Level
- Intermediate
- Format
- Book
- Cost
- Paid
- Duration
- ~500 pages across 10 chapters, self-paced reference reading
- Provider
- Chip Huyen (O'Reilly Media)
- Hands-on
- No
- Certificate
- None
What You’ll Learn
- ✓Decide whether a use case needs an agent, RAG, finetuning, or none of them
- ✓Build an evaluation pipeline that scores every component of an AI system independently
- ✓Read and interpret perplexity, cross-entropy and bits-per-byte when comparing models
- ✓Use AI-as-judge correctly and recognize where its judgements stop being reliable
- ✓Defend prompts against jailbreaking, injection and system-prompt extraction
- ✓Do the memory math that determines whether a given finetune fits your hardware
- ✓Design production architecture with guardrails, model routing, caching and observability
Highlights
- •Two full chapters on evaluation, the subject most AI books compress into a few pages
- •Written by the author of Designing Machine Learning Systems, with NVIDIA and Netflix production experience
- •Organised around the inverted workflow: product first, model selection second, training last
- •Free companion repo (chiphuyen/aie-book, 17k+ stars) with chapter summaries and a resource list
- •Deliberately written as a durable desk reference rather than a tour of this year's tools
Who It’s For
Best For
- ✓Software engineers moving into AI application work without an ML research background
- ✓ML engineers adapting from training their own models to building on foundation models
- ✓Tech leads who need shared vocabulary for evaluation, model selection and architecture
Prerequisites
- •Working software engineering experience; the book assumes you can read code
- •No prior deep learning or ML research background required
FAQ
What is AI Engineering: Building Applications with Foundation Models?
Chip Huyen's O'Reilly book on building applications with foundation models, covering evaluation, prompt engineering, RAG, agents, finetuning, dataset engineering, inference optimization and production architecture. Written for engineers who start from a product idea and an available model rather than a training run. Afterwards you can scope, evaluate and ship an AI feature and justify each architectural decision on evidence.
Is AI Engineering: Building Applications with Foundation Models free?
AI Engineering: Building Applications with Foundation Models is a paid resource.
What level is AI Engineering: Building Applications with Foundation Models for?
AI Engineering: Building Applications with Foundation Models is aimed at a intermediate audience. Recommended background: Working software engineering experience; the book assumes you can read code, No prior deep learning or ML research background required.
How long does AI Engineering: Building Applications with Foundation Models take?
Expect roughly ~500 pages across 10 chapters, self-paced reference reading. Most learners work through it at their own pace.
What will I learn from AI Engineering: Building Applications with Foundation Models?
You'll learn: Decide whether a use case needs an agent, RAG, finetuning, or none of them; Build an evaluation pipeline that scores every component of an AI system independently; Read and interpret perplexity, cross-entropy and bits-per-byte when comparing models; Use AI-as-judge correctly and recognize where its judgements stop being reliable; Defend prompts against jailbreaking, injection and system-prompt extraction; Do the memory math that determines whether a given finetune fits your hardware; Design production architecture with guardrails, model routing, caching and observability.
Topics
Sources
This page was written from 3 sources, 2 on domains other than huyenchip.com.