Toward Efficient Agents: Memory, Tool learning, and Planning
by Shanghai AI Laboratory and collaborators
A 63-page map of methods that cut agent token use, latency and step count across memory, tool calling and planning.
Overview
This survey from researchers at Shanghai Artificial Intelligence Laboratory, Shanghai Jiao Tong University, Fudan, Tsinghua and other institutions was first posted on January 20, 2026 and revised on July 3, 2026 (v2). It runs 63 pages with 244 references. After preliminaries that model an agent as a POMDP with tools and memory, and a cost model that adds tool, memory and retry costs to token generation, it has three main parts. Efficient Memory covers construction (latent and parametric memory such as MemoRAG and Titans; textual memory as prompt-resident, item-based, graph-based or hierarchical, with systems such as MemGPT, Mem0, A-MEM, Zep and MemoryOS), rule-based, LLM-based and hybrid management, memory selection and integration, skill reuse and multi-agent memory. Efficient Tool Use covers tool selection by retriever, classifier or vocabulary tokens; parallel and cost-aware calling such as LLMCompiler and budget-constrained planning; and RL post-training with efficiency rewards such as OTC-PO's tool-use penalty and PORTool's step-wise advantages. Efficient Planning covers adaptive budgeting, search such as LATS, task decomposition such as ReWOO, and multi-agent topology pruning. A benchmarks section pairs effectiveness suites (GAIA, SWE-Bench, tau-bench, LongMemEval, BFCL, MCP-Bench) with efficiency measures such as TPS-Bench's cost-of-pass and CostBench. A companion GitHub paper list is maintained alongside it.
At a Glance
- Topic
- Agentic
- Level
- Advanced
- Format
- Paper
- Cost
- Free
- Duration
- 63-page survey with 244 references; several hours to read closely, longer if you follow the linked paper list
- Provider
- Shanghai AI Laboratory and collaborators
- Hands-on
- No
- Certificate
- None
What You’ll Learn
- ✓Define agent efficiency as a Pareto trade-off between task success and token, latency and step cost
- ✓Compare prompt-resident, item-based, graph-based and hierarchical memory designs and their compression costs
- ✓Choose between rule-based, LLM-based and hybrid memory management for updating and evicting memories
- ✓Cut prompt size by selecting tools with retrievers, classifiers or tool-vocabulary tokens
- ✓Reduce latency with parallel and cost-aware tool calling such as compiler-style execution plans
- ✓Recognize RL reward designs that penalize redundant tool calls without lowering task accuracy
- ✓Apply adaptive budgeting, tree search and task decomposition to bound planning cost
- ✓Measure efficiency with token, monetary, runtime and tool-call metrics from benchmarks like TPS-Bench and CostBench
Highlights
- •Covers memory, tool use and planning as one cost problem, which most agent surveys treat separately
- •Names concrete methods in every subsection, so it works as a lookup table when you need a technique for a specific bottleneck
- •Separates effectiveness benchmarks from efficiency measurements and lists which benchmarks report cost at all
- •Companion repo yxf203/Awesome-Efficient-Agents had 315 stars when checked on October 31, 2026, with paper entries dated as recently as 2026-09
- •Revised in July 2026, so it includes 2026 work on agent skills and RL-trained tool policies
Who It’s For
Best For
- ✓AI engineers whose agents are too slow or too expensive per task in production
- ✓Researchers looking for related work on agent memory, tool learning or planning efficiency
- ✓Tech leads choosing a memory architecture or tool-routing strategy for a new agent system
- ✓Teams designing cost-aware evaluations for agents beyond accuracy alone
Prerequisites
- •Comfort reading ML research papers and following citations to the original methods
- •Working knowledge of LLM agents: tool calling, RAG-style retrieval and multi-step planning
- •Basic reinforcement learning vocabulary (policy, reward, advantage) for the post-training sections
FAQ
What is Toward Efficient Agents: Memory, Tool learning, and Planning?
A research survey for AI engineers and researchers who need LLM agents to cost less per task. It organizes recent work on efficient memory, tool selection and calling, and planning, defines efficiency as a cost and effectiveness Pareto trade-off, and lists the benchmarks and metrics used to measure token, time and tool-call cost.
Is Toward Efficient Agents: Memory, Tool learning, and Planning free?
Toward Efficient Agents: Memory, Tool learning, and Planning is free to access.
What level is Toward Efficient Agents: Memory, Tool learning, and Planning for?
Toward Efficient Agents: Memory, Tool learning, and Planning is aimed at a advanced audience. Recommended background: Comfort reading ML research papers and following citations to the original methods, Working knowledge of LLM agents: tool calling, RAG-style retrieval and multi-step planning, Basic reinforcement learning vocabulary (policy, reward, advantage) for the post-training sections.
How long does Toward Efficient Agents: Memory, Tool learning, and Planning take?
Expect roughly 63-page survey with 244 references; several hours to read closely, longer if you follow the linked paper list. Most learners work through it at their own pace.
What will I learn from Toward Efficient Agents: Memory, Tool learning, and Planning?
You'll learn: Define agent efficiency as a Pareto trade-off between task success and token, latency and step cost; Compare prompt-resident, item-based, graph-based and hierarchical memory designs and their compression costs; Choose between rule-based, LLM-based and hybrid memory management for updating and evicting memories; Cut prompt size by selecting tools with retrievers, classifiers or tool-vocabulary tokens; Reduce latency with parallel and cost-aware tool calling such as compiler-style execution plans; Recognize RL reward designs that penalize redundant tool calls without lowering task accuracy; Apply adaptive budgeting, tree search and task decomposition to bound planning cost; Measure efficiency with token, monetary, runtime and tool-call metrics from benchmarks like TPS-Bench and CostBench.
Topics
Sources
This page was written from 4 sources, 2 on domains other than arxiv.org.