RAGAgenticFrameworks

Production Agentic RAG Course: arXiv Paper Curator

by Jam with AI (Shirin Khosravi Jam and Shantanu Ladhwe)

IntermediateCourseFreeWeek 0 overview plus 7 build weeks, self-paced; the authors publish no hour estimate

Build an arXiv research assistant in seven stages, from Docker services and BM25 to hybrid search, tracing and a LangGraph agent.

Start LearningAdded Nov 9, 2026 · Updated Nov 9, 2026

Overview

The Production Agentic RAG Course is a free, MIT-licensed GitHub course in which you build arXiv Paper Curator, a research assistant that fetches academic papers, parses them and answers questions about them. It is Phase 1 of the Mother of AI project by Shirin Khosravi Jam and Shantanu Ladhwe, who describe themselves as AI practitioners with 17+ years of combined experience building production ML systems. Week 0 is an overview post, followed by seven build weeks, each with a notebook and a tagged release (week1.0 through week7.0) so you can check out any stage. Week 1 stands up FastAPI, PostgreSQL 16, OpenSearch 2.19, Airflow 3.0 and Ollama with Docker Compose. Week 2 adds an arXiv client with rate limiting and retries, Docling PDF parsing and daily Airflow DAGs. Week 3 builds BM25 keyword search in OpenSearch. Week 4 adds section-aware chunking, Jina embeddings and reciprocal rank fusion behind one search API. Week 5 generates answers locally through Ollama, with server-sent event streaming and a Gradio interface. Week 6 adds Langfuse tracing and Redis caching with latency and cost monitoring. Week 7 wraps the pipeline in a LangGraph agent with an out-of-domain guardrail, document grading, query rewriting and a Telegram bot. The Week 7 code landed in November 2025, and setup fixes were merged through April 2026.

At a Glance

Topic: RAG
Level: Intermediate
Format: Course
Cost: Free
Duration: Week 0 overview plus 7 build weeks, self-paced; the authors publish no hour estimate
Provider: Jam with AI (Shirin Khosravi Jam and Shantanu Ladhwe)
Hands-on: Yes — code/exercises
Certificate: None

What You’ll Learn

✓Stand up FastAPI, PostgreSQL, OpenSearch, Airflow and Ollama together with Docker Compose
✓Ingest arXiv papers daily with a rate-limited client, retries, Docling parsing and Airflow DAGs
✓Build BM25 keyword search in OpenSearch with filtering and relevance scoring
✓Add section-aware chunking and Jina embeddings, then merge keyword and vector results with RRF
✓Generate streaming answers from a local Ollama model and expose them through Gradio
✓Trace the RAG pipeline with Langfuse and monitor latency and cost alongside Redis caching
✓Orchestrate agentic RAG in LangGraph with document grading, query rewriting and domain guardrails
✓Connect the finished agent to users through a Telegram bot interface

Highlights

•Keyword search gets its own week, so you have a working BM25 baseline before Week 4 adds embeddings and hybrid retrieval.
•Tagged releases from week1.0 to week7.0 let you check out the code at the end of any stage and compare it with your own.
•The repo had 10.0k stars and 2.2k forks on November 9, 2026, and a March 2026 commit fixed five Week 1 bugs that broke a fresh clone. The newest commit on main then was dated April 5, 2026.
•Local development costs $0 by the repo's estimate, with optional cloud LLM APIs at about $2 to $5. Some companion Substack material is paid, including the full Week 7 post and video, while the notebooks and code stay in the free repo.
•No week is dedicated to retrieval or answer evaluation. Week 6 covers tracing, caching, latency and cost, so plan to add your own eval set.

Who It’s For

Best For

✓Software engineers who want to build an end-to-end RAG service with separate running components
✓AI/ML engineers moving from basic vector search to hybrid retrieval and agentic RAG
✓Data scientists who want practice with Docker, Airflow and production search infrastructure
✓Learners who want a portfolio project with ingestion, search, monitoring and a user interface

Prerequisites

•Good Python and general software programming knowledge, the authors' stated prerequisite
•Docker Desktop with Docker Compose, Python 3.12+ and the UV package manager
•A machine with 8GB+ of RAM and 20GB+ of free disk space
•A Jina API key from Week 4 and a Telegram bot token for Week 7

FAQ

What is Production Agentic RAG Course: arXiv Paper Curator?

A free, MIT-licensed GitHub course for AI engineers, software engineers and data scientists who want to build a RAG system as a set of running services. You finish with an arXiv research assistant that has scheduled ingestion, hybrid search, local LLM answers, Langfuse tracing, Redis caching and a LangGraph agent behind a Telegram bot.

Is Production Agentic RAG Course: arXiv Paper Curator free?

Production Agentic RAG Course: arXiv Paper Curator is free to access.

What level is Production Agentic RAG Course: arXiv Paper Curator for?

Production Agentic RAG Course: arXiv Paper Curator is aimed at a intermediate audience. Recommended background: Good Python and general software programming knowledge, the authors' stated prerequisite, Docker Desktop with Docker Compose, Python 3.12+ and the UV package manager, A machine with 8GB+ of RAM and 20GB+ of free disk space, A Jina API key from Week 4 and a Telegram bot token for Week 7.

How long does Production Agentic RAG Course: arXiv Paper Curator take?

Expect roughly Week 0 overview plus 7 build weeks, self-paced; the authors publish no hour estimate. Most learners work through it at their own pace.

What will I learn from Production Agentic RAG Course: arXiv Paper Curator?

You'll learn: Stand up FastAPI, PostgreSQL, OpenSearch, Airflow and Ollama together with Docker Compose; Ingest arXiv papers daily with a rate-limited client, retries, Docling parsing and Airflow DAGs; Build BM25 keyword search in OpenSearch with filtering and relevance scoring; Add section-aware chunking and Jina embeddings, then merge keyword and vector results with RRF; Generate streaming answers from a local Ollama model and expose them through Gradio; Trace the RAG pipeline with Langfuse and monitor latency and cost alongside Redis caching; Orchestrate agentic RAG in LangGraph with document grading, query rewriting and domain guardrails; Connect the finished agent to users through a Telegram bot interface.

Topics

agentic-raghybrid-searchlanggraphopensearchlangfusedocker

Sources

This page was written from 7 sources, 5 on domains other than github.com.