The Landscape of Agentic Reinforcement Learning for LLMs: A Survey
by Guibin Zhang et al. (arXiv preprint)
The 100-page map of how RL turns an LLM from a text generator into a decision-making agent, plus a maintained repo of every algorithm, environment and framework it names.
Overview
This 100-page survey, first posted 2 September 2025 and revised through April 2026, is the reference map for the shift away from RLHF-style preference tuning of a single-turn text generator toward reinforcement learning over LLMs treated as decision-making agents. Twenty-five authors from academic and industry labs — among them Guibin Zhang, Hejia Geng, Zhenfei Yin, Hongru Wang, Heng Ji, Jun Wang, Shuicheng Yan, Philip Torr and Lei Bai — synthesise more than 500 recent works into a deliberately twofold taxonomy. The first axis is capability-centric: planning, tool use, memory, reasoning, self-improvement and perception, each framed as a behaviour that reinforcement learning converts from a static, hand-engineered module into something adaptive and trainable. The second axis is task-centric, walking through search and deep-research agents, code generation, mathematical reasoning, GUI and web navigation, and multi-agent systems. Structurally the paper runs from an introduction through a preliminaries section formalising the move from conventional LLM RL to agentic RL, into the capability chapters, then tool integration and environment interaction, specialised applications, and finally open challenges and future directions. Its practical value is arguably less in the prose than in the tables: the official companion repository, Awesome-AgenticLLM-RL-Papers (around 1.9k stars), maintains the same material as browsable lists — 50-plus algorithms across the PPO, DPO and GRPO families annotated by year, objective type and key mechanism; 40-plus environments and benchmarks; and the RL and RLHF training frameworks each method used, each row linking to paper, code and released models.
At a Glance
- Topic
- Agentic
- Level
- Advanced
- Format
- Paper
- Cost
- Free
- Duration
- ~100 pages — 4-6 hours for a full read, or use it as a maintained reference index
- Provider
- Guibin Zhang et al. (arXiv preprint)
- Hands-on
- No
- Certificate
- None
What You’ll Learn
- ✓Distinguish agentic RL from conventional RLHF in terms of horizon, reward signal and environment
- ✓Navigate the PPO, DPO and GRPO algorithm families and what each is actually suited to
- ✓Map the six agentic capabilities — planning, tool use, memory, reasoning, self-improvement, perception — to concrete training methods
- ✓Choose an appropriate environment or benchmark from 40-plus documented options for your task domain
- ✓Compare how RL is applied differently to search agents, coding agents, GUI agents and multi-agent systems
- ✓Identify which training frameworks the published methods actually used, and their tradeoffs
- ✓Recognise the open problems — reward design, scalability, evaluation — where the field currently has no answer
Highlights
- •One of the few surveys where the companion GitHub repo is genuinely the better artifact — maintained, tabulated, and linked to code and weights
- •The twofold capability/task taxonomy lets you enter from either side: 'I need better tool use' or 'I'm building a web agent'
- •Covers 500-plus works across a field that produced most of them in under two years, which no reading list keeps up with
- •Revised five times through April 2026, so it is a living document rather than a snapshot that rotted on posting
- •Author list spans academic and industry labs, which shows in the coverage of both training theory and deployed systems
Who It’s For
Best For
- ✓Researchers entering agentic RL who need to know what has already been tried before proposing anything
- ✓ML engineers choosing an RL algorithm and environment for a tool-using or coding agent
- ✓Technical leaders assessing whether RL post-training is the right lever for their agent's failure mode
- ✓Anyone building an agent benchmark and needing the existing landscape first
Prerequisites
- •Working knowledge of reinforcement learning fundamentals — policy gradients, reward modelling, PPO
- •Familiarity with LLM post-training, including SFT and RLHF pipelines
- •Comfort reading a dense academic survey with heavy forward references
FAQ
What is The Landscape of Agentic Reinforcement Learning for LLMs: A Survey?
A 100-page survey synthesising more than 500 works on agentic reinforcement learning for LLMs — the shift from preference-tuning a single-turn generator to training a model that plans, calls tools, remembers and self-improves across a trajectory. Organised by capability and by task domain, with a companion GitHub repo of algorithms, benchmarks and frameworks. Read it to locate your problem in a field moving faster than any one lab can track.
Is The Landscape of Agentic Reinforcement Learning for LLMs: A Survey free?
The Landscape of Agentic Reinforcement Learning for LLMs: A Survey is free to access.
What level is The Landscape of Agentic Reinforcement Learning for LLMs: A Survey for?
The Landscape of Agentic Reinforcement Learning for LLMs: A Survey is aimed at a advanced audience. Recommended background: Working knowledge of reinforcement learning fundamentals — policy gradients, reward modelling, PPO, Familiarity with LLM post-training, including SFT and RLHF pipelines, Comfort reading a dense academic survey with heavy forward references.
How long does The Landscape of Agentic Reinforcement Learning for LLMs: A Survey take?
Expect roughly ~100 pages — 4-6 hours for a full read, or use it as a maintained reference index. Most learners work through it at their own pace.
What will I learn from The Landscape of Agentic Reinforcement Learning for LLMs: A Survey?
You'll learn: Distinguish agentic RL from conventional RLHF in terms of horizon, reward signal and environment; Navigate the PPO, DPO and GRPO algorithm families and what each is actually suited to; Map the six agentic capabilities — planning, tool use, memory, reasoning, self-improvement, perception — to concrete training methods; Choose an appropriate environment or benchmark from 40-plus documented options for your task domain; Compare how RL is applied differently to search agents, coding agents, GUI agents and multi-agent systems; Identify which training frameworks the published methods actually used, and their tradeoffs; Recognise the open problems — reward design, scalability, evaluation — where the field currently has no answer.
Topics
Sources
This page was written from 2 sources, 1 on domains other than arxiv.org.