Hybrid Reasoning Agents
by Stanford University, UIUC and Together AI (Zou et al.)
LLM agents that reason in latent space and act through text, trained with SFT then RL, with open code and 8B/14B checkpoints.
Overview
Hybrid Reasoning Agents (arXiv:2610.18861, submitted October 13, 2026) is a 45-page paper by Jiaru Zou, Yunzhe Qi, Sirui Chen, Federico Bianchi, Shang Zhu, Pan Lu, Jingrui He and James Zou, with affiliations at Stanford, UIUC and Together AI. It proposes HybridAgents, which alternate between latent reasoning (continuous hidden states, no tokens emitted) and ordinary text for tool use, within a single generation. A lightweight component called HybridLink handles the switch between modes, and a continuous working memory carries context across turns. Section 3 builds the agent (the hybrid mode interface and multi-turn generation over working memory). Section 4 trains it in two stages: HybridSFT, a self-conditioned supervised stage that teaches when to enter and exit latent mode, then HybridRL, which learns when to think latently and when to act in text. Sections 5 and 6 evaluate 4B, 8B and 14B models on Qwen3-Base across 11 benchmarks: AIME 25, AIME 26, MATH500, GPQA-Diamond, MedQA, MuSiQue, GAIA, HLE, WebWalker, SealQA and DeepSearchQA. The authors report a 10.4% average gain over frontier agents and agent frameworks, 35.5-54.7% fewer tokens, 1.8-2.6x faster inference and 2.7-3.7x lower cost. The 14B RL model scores 43.3 on AIME25 and 88.2 on MATH500. Appendices cover the RL policy factorization, data curation, latent interpretability and prompt templates. The MIT-licensed togethercomputer/HybridAgent repo has the training and inference scripts, with checkpoints and datasets on Hugging Face.
At a Glance
- Topic
- Agentic
- Level
- Advanced
- Format
- Paper
- Cost
- Free
- Duration
- ~2-3 hours to read the 45-page paper; reproducing training needs 8-32 GPUs
- Provider
- Stanford University, UIUC and Together AI (Zou et al.)
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓How an agent can switch between latent reasoning and text tool calls mid-generation
- ✓How the HybridLink module and a continuous working memory carry context across turns
- ✓How self-conditioned HybridSFT teaches a model when to enter and exit latent mode
- ✓How HybridRL factorizes the policy to learn when to think silently versus act
- ✓How to measure token, latency and cost savings against text-only agent baselines
- ✓How the authors audit latent reasoning and tool use for interpretability
- ✓How to run the released 8B and 14B checkpoints with the batch inference scripts
Highlights
- •Open release: MIT-licensed code, HybridSFT and HybridRL datasets, and 8B and 14B agentic checkpoints on Hugging Face
- •Evaluated on 11 benchmarks spanning math, science, medicine and agentic web search such as GAIA and HLE
- •Reports 35.5-54.7% fewer tokens and 2.7-3.7x lower cost, which matters for anyone paying per agent step
- •Drew 520 points and 239 comments on Hacker News, where practitioners questioned whether latent reasoning can still be monitored
- •The repo's default HybridRL setup is 32 GPUs (4 nodes x 8), so full reproduction is out of reach for most individuals
Who It’s For
Best For
- ✓Researchers and engineers working on agent post-training with SFT and reinforcement learning
- ✓Teams trying to cut token spend and latency on multi-step reasoning agents
- ✓Safety and observability engineers weighing the cost of losing a readable chain of thought
- ✓Readers following latent and continuous reasoning research beyond text chain-of-thought
Prerequisites
- •Working knowledge of transformer LLMs, supervised fine-tuning and RL for LLMs (GRPO/PPO-style policy optimization)
- •Familiarity with tool-using agents and benchmarks such as GAIA
- •To run the code: Python 3.11, PyTorch 2.11 with CUDA 12.8, GPUs, and a Tavily API key for the search tool
FAQ
What is Hybrid Reasoning Agents?
Hybrid Reasoning Agents is an October 2026 research paper from Stanford, UIUC and Together AI for agent and post-training engineers. It shows how to train an agent that switches between hidden latent reasoning and text tool calls inside one generation, and it ships code, data and Qwen3-based checkpoints you can run or retrain.
Is Hybrid Reasoning Agents free?
Hybrid Reasoning Agents is free to access.
What level is Hybrid Reasoning Agents for?
Hybrid Reasoning Agents is aimed at a advanced audience. Recommended background: Working knowledge of transformer LLMs, supervised fine-tuning and RL for LLMs (GRPO/PPO-style policy optimization), Familiarity with tool-using agents and benchmarks such as GAIA, To run the code: Python 3.11, PyTorch 2.11 with CUDA 12.8, GPUs, and a Tavily API key for the search tool.
How long does Hybrid Reasoning Agents take?
Expect roughly ~2-3 hours to read the 45-page paper; reproducing training needs 8-32 GPUs. Most learners work through it at their own pace.
What will I learn from Hybrid Reasoning Agents?
You'll learn: How an agent can switch between latent reasoning and text tool calls mid-generation; How the HybridLink module and a continuous working memory carry context across turns; How self-conditioned HybridSFT teaches a model when to enter and exit latent mode; How HybridRL factorizes the policy to learn when to think silently versus act; How to measure token, latency and cost savings against text-only agent baselines; How the authors audit latent reasoning and tool use for interpretability; How to run the released 8B and 14B checkpoints with the batch inference scripts.
Topics
Sources
This page was written from 4 sources, 2 on domains other than arxiv.org.