GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
by Agrawal et al. (UC Berkeley, Stanford)
Reflecting on traces in natural language beat GRPO with 35x fewer rollouts.
Overview
GEPA (Genetic-Pareto) is an ICLR 2026 oral paper from Lakshya A Agrawal and sixteen co-authors across UC Berkeley and Stanford, a group that includes Omar Khattab, Christopher Potts, Ion Stoica and Dan Klein. Its argument is that the interpretable nature of language is a richer learning medium for LLMs than policy gradients derived from sparse scalar rewards. Where GRPO often needs thousands of rollouts to learn a task, GEPA samples trajectories from the system being optimized — reasoning, tool calls, tool outputs — has a language model reflect on them in natural language to diagnose what went wrong, proposes and tests prompt updates, and combines complementary lessons from the Pareto frontier of its own attempts rather than tracking a single incumbent. Three mechanisms carry the method: reflective mutation, Pareto candidate selection, and a system-aware merge. The paper reports outperforming GRPO by 6% on average and by up to 20% across six tasks while using up to 35x fewer rollouts, and beating MIPROv2, the previously leading prompt optimizer, by over 10% including a 12% accuracy gain on AIME-2025; it also demonstrates promise as an inference-time search strategy for code optimization. Version 2, dated February 2026, is the ICLR camera-ready. The MIT-licensed reference implementation lives at gepa-ai/gepa.
At a Glance
- Topic
- Fine-Tuning
- Level
- Advanced
- Format
- Paper
- Cost
- Free
- Duration
- ~45-60 min read for the main paper; longer with the appendices
- Provider
- Agrawal et al. (UC Berkeley, Stanford)
- Hands-on
- No
- Certificate
- None
What You’ll Learn
- ✓Why textual reflection can be a richer learning signal than scalar policy gradients
- ✓How reflective mutation turns raw execution traces into targeted prompt edits
- ✓Why a Pareto frontier of candidates beats greedy single-incumbent prompt search
- ✓How a system-aware merge combines lessons from separate optimization lineages
- ✓When prompt optimization is a cheaper substitute for GRPO-style reinforcement learning
- ✓How to read the rollout-efficiency argument: comparable gains at 35x fewer rollouts
- ✓How the same method extends to inference-time search for code optimization
Highlights
- •Accepted at ICLR 2026 as an oral, the top tier of acceptance; the February 2026 v2 is the camera-ready
- •Benchmarks prompt optimization directly against RL fine-tuning (GRPO), the comparison most prompt-optimization papers avoid making
- •The reference implementation is MIT-licensed, actively developed at roughly 6k stars, and exposed as dspy.GEPA
- •Ships adapters for DSPy, LangChain, Pydantic AI, Google ADK and even MCP tool descriptions, so it runs on a real stack today
- •Written by the DSPy lineage (Khattab, Potts) alongside systems researchers such as Ion Stoica, so the efficiency claims are framed in rollout cost
Who It’s For
Best For
- ✓Engineers weighing prompt optimization against fine-tuning for an LLM system
- ✓DSPy users looking for a stronger optimizer than MIPROv2
- ✓Researchers working on LLM self-improvement and agent optimization
- ✓Teams with constrained rollout or GPU budgets
Prerequisites
- •Understanding of LLM prompting and compound AI system design
- •Familiarity with RL post-training basics, particularly GRPO or PPO
- •Comfort reading an ML research paper with benchmark tables
FAQ
What is GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning?
The ICLR 2026 oral paper introducing GEPA (Genetic-Pareto), a prompt optimizer that reads an LLM system's own execution traces, diagnoses failures in natural language, and evolves prompts along a Pareto frontier. It reports beating GRPO by 6% on average across six tasks while using up to 35x fewer rollouts, making it the reference argument for optimizing prompts before reaching for reinforcement learning.
Is GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning free?
GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning is free to access.
What level is GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning for?
GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning is aimed at a advanced audience. Recommended background: Understanding of LLM prompting and compound AI system design, Familiarity with RL post-training basics, particularly GRPO or PPO, Comfort reading an ML research paper with benchmark tables.
How long does GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning take?
Expect roughly ~45-60 min read for the main paper; longer with the appendices. Most learners work through it at their own pace.
What will I learn from GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning?
You'll learn: Why textual reflection can be a richer learning signal than scalar policy gradients; How reflective mutation turns raw execution traces into targeted prompt edits; Why a Pareto frontier of candidates beats greedy single-incumbent prompt search; How a system-aware merge combines lessons from separate optimization lineages; When prompt optimization is a cheaper substitute for GRPO-style reinforcement learning; How to read the rollout-efficiency argument: comparable gains at 35x fewer rollouts; How the same method extends to inference-time search for code optimization.
Topics
Sources
This page was written from 2 sources, 1 on domains other than arxiv.org.
- 1.arxiv.org — 2507.19457vendor
- 2.github.com — gepa