Building and Evaluating Advanced RAG Applications
by DeepLearning.AI × LlamaIndex & TruEra
Two retrieval upgrades that beat naive RAG, plus the eval loop that proves it.
Overview
A 2-hour-5-minute DeepLearning.AI short course co-taught by Jerry Liu, co-founder and CEO of LlamaIndex, and Anupam Datta, who built TruEra and is now AI Research Lead at Snowflake. It is seven items: Introduction (4m), Advanced RAG Pipeline (15m), RAG Triad of metrics (42m), Sentence-window retrieval (29m), Auto-merging retrieval (21m), Conclusion (1m) and a 10-minute graded quiz — four of them runnable notebooks, mirrored publicly on GitHub as L1 through L4. The longest lesson is the evaluation one, which is the whole point: it teaches the 'RAG triad' of Context Relevance (did retrieval return the right chunks), Groundedness (is the answer supported by those chunks) and Answer Relevance (does it actually address the question), instrumented with TruLens so every change to the pipeline produces a score rather than a vibe. The two retrieval techniques are the standard first upgrades over naive chunking. Sentence-window retrieval embeds single sentences for precise matching but feeds the model the surrounding window of sentences for context. Auto-merging retrieval builds a hierarchy in which a parent node's text is split across children, and when enough children match a query, the parent's full text is substituted. It is deliberately narrow: two hours, no capstone, no certificate, and tightly coupled to LlamaIndex and TruLens rather than framework-agnostic theory.
At a Glance
- Topic
- RAG
- Level
- Intermediate
- Format
- Course
- Cost
- Free
- Duration
- 2 hrs 5 min, self-paced
- Provider
- DeepLearning.AI × LlamaIndex & TruEra
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Why naive chunk-and-embed RAG fails and where the answer quality is lost
- ✓Sentence-window retrieval: embed sentences, serve the surrounding context window
- ✓Auto-merging retrieval over a parent-child hierarchy of document chunks
- ✓The RAG triad: context relevance, groundedness and answer relevance
- ✓Instrumenting a LlamaIndex pipeline with TruLens to score every experiment
- ✓Running an evaluate-change-re-evaluate loop instead of eyeballing outputs
Highlights
- •Co-taught by LlamaIndex's founder and the person who built TruEra's eval stack
- •The 42-minute evaluation lesson is longer than either retrieval lesson, deliberately
- •All four lesson notebooks are mirrored publicly on GitHub and runnable
- •Evaluation-first framing is the part almost every other RAG tutorial skips
- •Two hours total, free, and closed out by a graded quiz
Who It’s For
Best For
- ✓Developers whose RAG demo works but answers wrongly on real questions
- ✓Engineers who need a defensible quality metric for a RAG system
- ✓Teams already on LlamaIndex looking for the next retrieval upgrade
Prerequisites
- •Basic Python and comfort working in a Jupyter notebook
- •Familiarity with embeddings, vector stores and a basic RAG pipeline
FAQ
What is Building and Evaluating Advanced RAG Applications?
A two-hour DeepLearning.AI short course from LlamaIndex's Jerry Liu and TruEra's Anupam Datta on the two things naive RAG gets wrong: retrieval quality and measurement. You implement sentence-window and auto-merging retrieval, then instrument the pipeline with TruLens to score context relevance, groundedness and answer relevance. You finish able to iterate on a RAG system with data instead of intuition.
Is Building and Evaluating Advanced RAG Applications free?
Building and Evaluating Advanced RAG Applications is free to access.
What level is Building and Evaluating Advanced RAG Applications for?
Building and Evaluating Advanced RAG Applications is aimed at a intermediate audience. Recommended background: Basic Python and comfort working in a Jupyter notebook, Familiarity with embeddings, vector stores and a basic RAG pipeline.
How long does Building and Evaluating Advanced RAG Applications take?
Expect roughly 2 hrs 5 min, self-paced. Most learners work through it at their own pace.
What will I learn from Building and Evaluating Advanced RAG Applications?
You'll learn: Why naive chunk-and-embed RAG fails and where the answer quality is lost; Sentence-window retrieval: embed sentences, serve the surrounding context window; Auto-merging retrieval over a parent-child hierarchy of document chunks; The RAG triad: context relevance, groundedness and answer relevance; Instrumenting a LlamaIndex pipeline with TruLens to score every experiment; Running an evaluate-change-re-evaluate loop instead of eyeballing outputs.
Topics
Sources
This page was written from 2 sources, 1 on domains other than deeplearning.ai.