AI Engineer Notebooks
by calm.rocks
Build RAG, agent loops, evals and serving math from raw API calls in free Colab notebooks that run on Groq.
Overview
AI Engineer Notebooks is an open curriculum published on GitHub by calm.rocks in August 2026 as a companion to its guide on moving into Forward Deployed Engineer and AI Engineer roles. It is organized into 13 sections. Section 00 covers setup, API keys, cost monitoring and model choice. Section 01 (Model APIs, 5 notebooks) covers prompting, structured output, tool calling, streaming and prompt caching. Section 02 introduces golden-dataset evals, and Section 03 (RAG, 5 notebooks) moves from a 15-line RAG demo through embeddings, hybrid retrieval, reranking, chunking messy data and diagnosing failures. Section 04 adds RAG golden sets, LLM-as-judge and regression tests as CI. Section 05 (Agents, 6 notebooks) builds an agent loop from scratch, then covers tool design, cost and latency budgets, MCP, skills and harness engineering. Later sections cover fine-tuning versus RAG versus prompting with LoRA and QLoRA, prompt injection and the OWASP LLM Top 10, tracing and retries with MLflow, serving trade-offs across vLLM, TGI, Triton and TensorRT-LLM, inference system design, customer discovery, and four case studies including a contract-extraction comparison of an agent against a fixed pipeline and a PAIR red-team loop. Everything runs on the free Groq API without frameworks; GPU-dependent topics are taught concept-first with an optional appendix.
At a Glance
- Topic
- RAG
- Level
- Intermediate
- Format
- Course
- Cost
- Free
- Duration
- 33 Colab notebooks across 13 sections, self-paced (no official time estimate)
- Provider
- calm.rocks
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Implement tool calling, structured outputs and streaming directly against a model API
- ✓Build a RAG pipeline with embeddings, hybrid retrieval, reranking and chunking strategies
- ✓Diagnose RAG failures and trace most of them back to retrieval rather than generation
- ✓Write golden datasets, LLM-as-judge graders and regression tests that run in CI
- ✓Code an agent loop from scratch and design tools the model can use reliably
- ✓Choose between fine-tuning, RAG and prompting, and estimate LoRA and QLoRA costs
- ✓Demonstrate direct and indirect prompt injection and map risks to the OWASP LLM Top 10
- ✓Size an inference deployment for QPS, VRAM, latency and cost using napkin math
Highlights
- •Framework-free by design: agent loops, RAG and eval harnesses are written from raw API calls before any library is introduced
- •Runs end to end on the free Groq API with no credit card, and every notebook installs its own dependencies
- •Covers job-shaped skills most courses skip, including ML system design interviews, customer discovery and scoping
- •675 GitHub stars and 60 forks within weeks of its August 11, 2026 release (checked October 2, 2026); listed in KDnuggets' September 29, 2026 roundup of free AI engineering courses
Who It’s For
Best For
- ✓Backend or full-stack engineers moving into AI Engineer or Forward Deployed Engineer roles
- ✓Developers who want to understand RAG and agents before adopting LangChain or LlamaIndex
- ✓Candidates preparing for applied AI and ML system design interviews
- ✓Engineers who need a portfolio capstone with a deployed service and an eval report
Prerequisites
- •Comfortable writing and shipping production Python code
- •A free Groq API key and a Google Colab account or local Jupyter install
- •Optional GPU access for the LoRA fine-tuning appendix
FAQ
What is AI Engineer Notebooks?
AI Engineer Notebooks is a free, MIT-licensed set of Colab notebooks for backend and full-stack engineers moving into AI Engineer, Applied AI or Forward Deployed Engineer roles. You build RAG pipelines, agent loops, eval harnesses and security tests from raw model API calls, and finish with a deployed capstone repo and an evaluation report.
Is AI Engineer Notebooks free?
AI Engineer Notebooks is free to access.
What level is AI Engineer Notebooks for?
AI Engineer Notebooks is aimed at a intermediate audience. Recommended background: Comfortable writing and shipping production Python code, A free Groq API key and a Google Colab account or local Jupyter install, Optional GPU access for the LoRA fine-tuning appendix.
How long does AI Engineer Notebooks take?
Expect roughly 33 Colab notebooks across 13 sections, self-paced (no official time estimate). Most learners work through it at their own pace.
What will I learn from AI Engineer Notebooks?
You'll learn: Implement tool calling, structured outputs and streaming directly against a model API; Build a RAG pipeline with embeddings, hybrid retrieval, reranking and chunking strategies; Diagnose RAG failures and trace most of them back to retrieval rather than generation; Write golden datasets, LLM-as-judge graders and regression tests that run in CI; Code an agent loop from scratch and design tools the model can use reliably; Choose between fine-tuning, RAG and prompting, and estimate LoRA and QLoRA costs; Demonstrate direct and indirect prompt injection and map risks to the OWASP LLM Top 10; Size an inference deployment for QPS, VRAM, latency and cost using napkin math.
Topics
Sources
This page was written from 4 sources, 3 on domains other than github.com.