A new arXiv paper landed this week with a counterintuitive finding that should be on the desk of every AI engineering leader: the same reinforcement learning training that makes a model better at reasoning makes it more likely to fabricate tools that do not exist. As reasoning capability goes up, tool hallucination goes up with it — proportionally, causally, and across every mitigation the authors tested.
The paper is "The Reasoning Trap: How Enhancing LLM Reasoning Amplifies Tool Hallucination" (arXiv:2510.22977). It was submitted to ICLR 2026 in Rio de Janeiro and is now under discussion on OpenReview. The exact reception arc matters less than the finding: a sober, empirical claim that the dominant recipe the entire industry is using to make agents "smarter" is also making them less reliable when they hold the keys to your enterprise systems.
This collides with a brutal piece of market reality. According to OutSystems' 2026 State of AI Development report covering roughly 1,900 IT leaders, 96% of enterprises run AI agents in some form, and 94% report concern about sprawl, complexity and security risk. Deloitte's 2026 State of AI in the Enterprise research found that 47% of enterprise AI users have based at least one major business decision on hallucinated content. The gap between "we are deploying agents at scale" and "we trust the tools they call" has never been wider.
If you ship reasoning-model-based agents into production for any enterprise workflow that touches a real API — and most of you do — this paper changes how you should think about the deployment.
The Finding, in Plain Terms
The authors built a benchmark called SimpleToolHalluBench. It strips away every distraction and tests one specific question: when an agent is asked to do something it cannot do, does it correctly refuse, or does it invent a tool that does not exist?
The benchmark presents two failure modes:
-
No tools available. The agent has access to zero tools but is given a task. The reliable behavior is to say "I cannot do that with the tools available." The hallucinated behavior is to call a fictional tool like
search_database()orsend_email(). -
Only distractor tools. The agent has access to a tool palette, but none of the tools available are appropriate for the task. The reliable behavior is to say so. The hallucinated behavior is to call a wrong tool while pretending it solves the problem.
These are not edge cases. These are exactly the conditions every production AI agent encounters multiple times per session: missing permissions, deprecated endpoints, misconfigured tool registries, schema mismatches between tool catalog and actual deployment.
The paper's central claim, after running this benchmark across a range of open and closed models, is that reasoning enhancement via RL increases tool hallucination proportionally with the gain in task performance. The harder you train the model to think, the more it confabulates tools. The relationship is not noise. It is a consistent, measurable trade-off.
Worse, the effect is method-agnostic. The authors show it surfaces in three distinct conditions:
- When reasoning is taught via supervised fine-tuning on chain-of-thought traces.
- When reasoning is instilled via RL with verifiable rewards (the post-2024 dominant recipe).
- When reasoning is merely elicited at inference — the model is the same, but you flip the prompt from "answer directly" to "think step by step."
That third one is the killer. It means a vanilla model that already exhibits the behavior at low frequency will hallucinate tools more often the moment you turn on extended thinking, even without any training change at all. Every enterprise that A/B-tested reasoning mode and saw better task scores may have shipped a regression in tool reliability they never measured.
Why This Is Not Just Overfitting
The most natural objection — "this is a benchmark artifact, the models overfit to test-set patterns" — does not survive the paper's design.
The authors show the effect transfers across distinct task domains. Training reasoning capability on pure mathematics tasks (no tools, no agents, no APIs) still amplifies tool hallucination on subsequent agentic evaluations. The model never saw a tool during training. It got better at reasoning. And it became less reliable at refusing fictional tool calls.
That points at something deeper than memorization. The mechanistic analysis in the paper makes the structural claim explicit: reasoning RL disproportionately collapses tool-reliability-related representations. The paper traces hallucinated tool calls to amplified divergences concentrated in late-layer residual streams — the same layers that govern whether the model commits to an action versus declines.
In other words, the layers that should restrain a bad tool call get trained away as a side effect of optimizing for reasoning depth. The model becomes more confident, more willing to act, more likely to "complete" the task in some form — including by inventing the means to complete it. Confidence and confabulation, in this regime, are coupled.
This is a familiar pattern from a different angle. Anthropic's interpretability work, OpenAI's guardrails research, and a growing body of mechanistic interpretability literature have all flagged that the circuits which mediate refusal, calibration and abstention are fragile and easily overwritten by capability training. The Reasoning Trap paper is the cleanest demonstration to date that the same dynamic governs tool reliability.
The Mitigation Gap
The authors do not stop at the diagnosis. They evaluate the standard fixes the field has converged on, and report results that should change every AI engineering roadmap:
- Prompt engineering (system prompts that instruct the agent to refuse impossible tasks) yields modest improvement. It does not close the gap.
- Direct Preference Optimization (DPO) trained against preference data favoring refusal over hallucination yields larger improvement. It also does not close the gap.
- Across both, the authors document a fundamental reliability-capability trade-off: every intervention that meaningfully reduced tool hallucination also degraded task performance on legitimate work.
That trade-off is the part that should restructure your engineering planning. The framing inside most AI teams today is: "we will get better reasoning capability, then we will fix hallucinations on top." This paper's findings argue the framing is wrong. Reasoning capability and tool reliability are not stacked. They are in tension, and current methods do not jointly optimize them.
It implies one of three engineering responses:
-
Accept the trade-off explicitly. Configure agents in a high-reliability / lower-capability mode for production tool use, and a high-capability / human-supervised mode for analysis. This is the route Anthropic-style "constitutional" guardrails and OpenAI's structured tool-call gating already pushes toward, but most enterprise deployments have not formalized.
-
Move tool reliability out of the model. Treat the model as untrustworthy on tool selection by default and put the reliability work in the agent runtime: strict tool registries, signed tool catalogs, schema validation that rejects calls to non-existent endpoints, planner-executor architectures where the planner cannot directly invoke fabricated tools.
-
Wait for new training methods. A research bet that the field will produce reasoning training that does not collapse refusal circuits. Plausible. Not safe to plan deployments around.
The honest answer for most enterprises is some combination of (1) and (2) over the next 12 months, with (3) as a watching brief.
Why This Lands Hardest in Enterprise Workflows
Consumer chatbots tolerate tool hallucination more than enterprise systems do. If a chat assistant confabulates a fictional restaurant API, the user gets a weird answer and tries again. The cost is annoyance.
Enterprise agents do not have that buffer. The workflows under heaviest agent deployment in 2026 — HR (recruiting, onboarding, payroll, benefits), IT operations (ticketing, identity, provisioning), customer service (CRM, billing, returns), finance (reconciliation, AP/AR, expense), security operations (SIEM, EDR, ticketing) — all run on tool calls the agent must get exactly right. Every one of these surfaces is full of:
- APIs that exist but are deprecated.
- APIs that exist but the agent does not have permission to call.
- APIs that look like they should exist by name analogy but do not.
- APIs that exist with subtly different schemas across tenants.
This is exactly the territory the Reasoning Trap paper warns about. A reasoning-enhanced agent in this environment is more likely to invent a get_employee_record_by_ssn() call when none exists, more likely to pretend a returned 403 response was a 200, more likely to construct a plausible-looking tool name and ship a hallucinated API call into a downstream system.
The Deloitte data point — 47% of enterprise users have based at least one major business decision on hallucinated content — is a description of where this leads when reliability is left in the model.
A Concrete Engineering Checklist
If you own AI agent infrastructure, the paper's findings translate into actions you can take this quarter without waiting for the next training breakthrough:
Tool registry as a security boundary. Treat the tool registry the same way you treat IAM. Sign the catalog. Validate every tool name on the call path against the signed registry before execution. Reject calls to tool names not in the registry as hallucinations, not as 404 errors. Log them as a security event.
Schema-side enforcement. Do not trust the model to format tool calls correctly. Wrap every tool call in a schema validator that fails closed on unknown fields, type mismatches, or out-of-range values. This is table stakes that many production agent frameworks still skip.
Reasoning depth as a configuration, not a default. If your platform exposes "extended thinking" or "high-reasoning mode" to enterprise users, instrument tool-hallucination rates on it explicitly. Treat reasoning mode the way you treat a feature flag with security implications. Some workflows benefit; others regress. Measure per workflow.
Refusal-trained mitigation layer. Stand up a small DPO-trained refusal layer in front of high-stakes tool calls. The Reasoning Trap paper shows DPO does not close the gap — but it narrows it, and in regulated workflows the residual gap can be carried by humans, not by the model.
Eval suite that tests the failure mode the paper identifies. Build internal versions of SimpleToolHalluBench scenarios for your own tool catalog: tasks with no available tools, tasks with only distractor tools, tasks where a plausible-sounding-but-fictional tool name would solve the problem. Run them on every model upgrade and every reasoning-mode change. Track hallucination rate as a first-class production metric alongside latency and accuracy.
Planner-executor separation. Architect agents so the component that decides what to do is structurally incapable of invoking fabricated tools. The planner produces a plan in a constrained DSL. The executor maps planned actions to registry-validated tool calls. This is more engineering effort than letting the model emit raw function calls, and it is what survives in production when the model gets weirder under reasoning training.
What CIOs and CISOs Should Require
Pull the technical detail up to the policy layer:
Tool call provenance in audit logs. Every tool call attempted by an agent — successful, failed, or rejected by the registry — needs to be logged with the model identity, the prompt, the tool name attempted, and the outcome. This is the artifact your future incident response will live or die on.
Vendor disclosure on reasoning training. When evaluating an agentic AI vendor, the question to ask is not just "what model is under the hood." It is: "what RL training has been applied to the reasoning chain, and what tool-hallucination benchmarks did you evaluate against?" Vendors that cannot answer the second question are shipping reasoning-trap risk by default.
A tested rollback path on reasoning mode. If turning reasoning mode on regresses tool reliability, you need to be able to turn it off without redeploying the application. Configure the toggle, exercise it once a quarter, document the operational impact.
Risk allocation for the 47% number. The Deloitte finding that almost half of enterprise AI users have made a major decision on hallucinated content is not someone else's problem. It is a description of your organization unless you can show otherwise. The current control set in most enterprises was not designed for tool hallucination as a failure mode. It needs to be.
Bottom Line
The Reasoning Trap paper does not say reasoning models are unusable. It says the dominant recipe for making them smarter has a built-in reliability cost on the exact dimension that matters most for enterprise agents: do they call real tools, or do they make tools up.
For AI engineering leaders, the path forward is to stop treating reasoning capability and tool reliability as the same axis, instrument the trade-off explicitly, and move the reliability work into the runtime where you can control it — registries, schemas, planner-executor separation, refusal layers, eval suites that test the failure mode directly.
For the 96% of enterprises already running agents, the paper is a deadline notice on the architectural decisions you have been postponing. The next 12 months are when the gap between "we deployed an agent" and "we deployed an agent we can defend" gets either closed or audited.
Pick the one you want to be on the other side of.
