Recursive Language Models
by MIT CSAIL (Alex L. Zhang, Tim Kraska, Omar Khattab)
The MIT paper that lets an LLM hold a multi-million-token prompt as a Python variable and recursively call itself over the pieces.
Overview
Recursive Language Models (arXiv:2512.24601) is a research paper by Alex L. Zhang, Tim Kraska and Omar Khattab of MIT CSAIL, first posted on 31 December 2025 and revised on 28 January and 11 May 2026 (v3: 9 pages of main text, 43 pages with the appendix). It frames arbitrarily long prompts as an inference-time scaling problem. Instead of feeding the prompt to the network, a Recursive Language Model initialises a Python REPL with the prompt stored as a variable; the root model sees only metadata such as its length and prefix, writes code to inspect, slice and search the text, and calls itself or a cheaper sub-model through an llm_query() function inside loops, so both input and output can exceed the context window. The paper walks through the method, a long-context task suite spanning constant, linear and quadratic complexity (S-NIAH up to 2^20 tokens, BrowseComp-Plus over 1,000 documents and 6-11M tokens, OOLONG, the new OOLONG-Pairs, CodeQA and LongCoT-mini), baselines (base models, a compaction/summary agent, CodeAct with BM25 retrieval and sub-calls, OpenCode and Claude Code), results, trajectory analyses, related work and limitations, with appendices on training details, negative results, prompts, and runtime and cost. With GPT-5 as the root model the authors report median gains of 26% over compaction, 130% over CodeAct with sub-calls and 13% over Claude Code at comparable cost, and they post-train a small natively recursive model, RLM-Qwen3-8B, on 1,000 filtered trajectories from Qwen3-Coder-480B. An MIT-licensed reference implementation, the rlms Python library, is published on GitHub alongside a minimal implementation and documentation.
At a Glance
- Topic
- Agentic
- Level
- Intermediate
- Format
- Paper
- Cost
- Free
- Duration
- ~1.5 hour read for the 9-page main paper; ~3-4 hours including the full 43-page appendix
- Provider
- MIT CSAIL (Alex L. Zhang, Tim Kraska, Omar Khattab)
- Hands-on
- No
- Certificate
- None
What You’ll Learn
- ✓How an RLM loads the whole prompt into a Python REPL variable instead of the context window
- ✓How the root model writes code to peek at, slice and search the input using only metadata about it
- ✓How llm_query() sub-calls let the model recursively invoke itself or a cheaper sub-model over chunks inside loops
- ✓How recursion depth, the choice of sub-call model (GPT-5 root with GPT-5-mini sub-calls) and batching drive quality and cost
- ✓How RLMs compare against compaction agents, CodeAct with BM25 retrieval, OpenCode and Claude Code on long-context tasks
- ✓How to benchmark long-context systems across constant, linear and quadratic complexity tasks such as S-NIAH, OOLONG, OOLONG-Pairs, BrowseComp-Plus and CodeQA
- ✓How RLM-Qwen3-8B was post-trained with rejection fine-tuning on 1,000 filtered trajectories from Qwen3-Coder-480B
- ✓Where RLMs still break: runaway sub-call costs, sequential latency, and confusing final answers with intermediate thoughts
Highlights
- •Ships with an official MIT-licensed library (pip install rlms, ~5.6k GitHub stars) that swaps llm.completion() for rlm.completion() and runs code in local, Docker, Modal, E2B, Daytona or Prime sandboxes with OpenAI, Anthropic, OpenRouter or vLLM backends
- •Striking results on quadratic-complexity aggregation: on OOLONG-Pairs, GPT-5 and Qwen3-Coder score at most 0.1% F1 when prompted directly but 58.0% and 23.1% F1 when run as depth-1 RLMs
- •Honest cost accounting and a dedicated negative-results appendix: median RLM runs are cheaper than median base-model runs, but long-tail outliers raise the average
- •Widely debated by practitioners: 161 points on Hacker News in January 2026, where commenters argued whether it is 'just subagents' and whether depth-1 recursion deserves the name, useful context for reading the claims critically
- •Actively maintained: three arXiv revisions between December 2025 and May 2026, and the repository includes trajectory visualisation plus a training harness that integrates with Prime Intellect's prime-rl
Who It’s For
Best For
- ✓AI engineers building agents that must reason over entire codebases, document dumps or very long logs
- ✓RAG practitioners evaluating whether model-driven decomposition can replace a fixed chunk-and-retrieve pipeline
- ✓Researchers working on inference-time scaling, long-context evaluation or context management for agents
Prerequisites
- •Comfort with LLM APIs and agent loops, including code-executing agents such as CodeAct
- •Working Python to follow the REPL-based trajectories and run the reference library (Python 3.11+)
- •Basic familiarity with RAG and long-context benchmarks helps but is not required
FAQ
What is Recursive Language Models?
Recursive Language Models is an MIT CSAIL research paper (Dec 2025, revised May 2026) for AI engineers fighting context-window limits. It shows how an LLM can keep a huge prompt as a variable inside a Python REPL and write code that recursively calls itself over chunks. Afterwards you can prototype RLM-style long-context agents with the authors' open-source rlms library and judge when the approach beats RAG or context compaction.
Is Recursive Language Models free?
Recursive Language Models is free to access.
What level is Recursive Language Models for?
Recursive Language Models is aimed at a intermediate audience. Recommended background: Comfort with LLM APIs and agent loops, including code-executing agents such as CodeAct, Working Python to follow the REPL-based trajectories and run the reference library (Python 3.11+), Basic familiarity with RAG and long-context benchmarks helps but is not required.
How long does Recursive Language Models take?
Expect roughly ~1.5 hour read for the 9-page main paper; ~3-4 hours including the full 43-page appendix. Most learners work through it at their own pace.
What will I learn from Recursive Language Models?
You'll learn: How an RLM loads the whole prompt into a Python REPL variable instead of the context window; How the root model writes code to peek at, slice and search the input using only metadata about it; How llm_query() sub-calls let the model recursively invoke itself or a cheaper sub-model over chunks inside loops; How recursion depth, the choice of sub-call model (GPT-5 root with GPT-5-mini sub-calls) and batching drive quality and cost; How RLMs compare against compaction agents, CodeAct with BM25 retrieval, OpenCode and Claude Code on long-context tasks; How to benchmark long-context systems across constant, linear and quadratic complexity tasks such as S-NIAH, OOLONG, OOLONG-Pairs, BrowseComp-Plus and CodeQA; How RLM-Qwen3-8B was post-trained with rejection fine-tuning on 1,000 filtered trajectories from Qwen3-Coder-480B; Where RLMs still break: runaway sub-call costs, sequential latency, and confusing final answers with intermediate thoughts.
Topics
Sources
This page was written from 6 sources, 4 on domains other than arxiv.org.