Turning on persistent memory made every configuration tested less accurate than turning it off. Not stale memory. Not hallucinated memory. Memory that was stored faithfully, retrieved correctly, and genuinely relevant to the question being asked.
That is the finding in MemTrapBench, submitted to arXiv on 20 August 2026 and surfaced sixth on the Hugging Face daily papers list the next day with 31 upvotes. Five memory frameworks, two model families, one control condition — no memory at all. The control won both averages and 39 of the 40 subset scores. On Gemini-3-Flash-Preview the no-memory baseline averaged 85.16%; the best memory framework managed 71.17% and the worst 54.69%, per the full results table in the paper. The authors' own summary is blunt: "all evaluated memory strategies underperform the no-memory setting, with even the strongest methods suffering drops of more than 10%."
If you are about to switch memory on across an assistant rollout or an agent fleet, this changes one thing in your plan, and it is not a big thing. Memory-off stops being the obvious downgrade you skip in the eval. It becomes the control arm.
What MemTrapBench Actually Measured
MemTrapBench measures what a correct memory does to the model's reasoning on the next task — the thing every other memory benchmark skips. A memory-induced cognitive trap is a stored fact that is accurate, relevant and correctly retrieved, and that still degrades the answer, because it anchors the model to the framing of the conversation it came from.
The benchmark holds 1,050 instances across four trap scenarios, built in three stages described in the paper's HTML edition: "(1) Plant the trap: a contextual prior is introduced in a plausible setting and repeatedly applied in subsequent interactions; (2) Bury It in Noise: unrelated turns are inserted, producing dialogues of 18–40 turns; and (3) Spring the Trap: the final query remains semantically related to the history but changes the conditions under which the prior should be applied."
Two failure families. Reasoning Fixation is when memory anchors the model to a previously established reasoning pattern — it covers Task Boundary (350 instances), Cognitive Bias (350) and Trauma (150). Belief Distortion is when memory changes what the model treats as true, tested in the Safety subset (200 instances) where a sandbox-specific or counterfactual premise from the history gets applied to a real-world query.
Read those definitions as a description of your Tuesday. Task Boundary is the user who spent forty minutes on a Q3 forecast and then asked an unrelated compliance question in the same thread. Trauma is the assistant that stops proposing an approach because someone rejected it once, three weeks ago, for a reason that no longer holds. Safety is the model that carries a "for this test environment, assume X" instruction into production advice.
The damage is worst exactly where you would expect. On Gemini, Task Boundary fell from 87.08% with no memory to 47.01% with full history in context. Safety fell from 95.90% to 56.15% under MemOS.
The Frameworks With the Best Published Scores Took the Worst Falls
The rank order on MemTrapBench is close to inverted from the rank order on the benchmarks these frameworks were built to win. That is the part worth sitting with.
SimpleMem reports a 26.4% average F1 improvement on LoCoMo and up to 30x fewer inference-time tokens. It scored 54.69% on MemTrapBench with Gemini — a 30-point drop against no memory, the worst of the five. EverMemOS, which claims 92.3% on LoCoMo in a December 2025 release and ships Apache 2.0 on GitHub with a Docker stack of MongoDB, Elasticsearch, Milvus and Redis, was the best performer here at 71.17% — still 14 points below doing nothing. LightMem claims up to 7.7% and 29.3% QA accuracy gains with 38x and 20.9x token reductions on LongMemEval and LoCoMo — those pairs are its GPT and Qwen backbones, not the two benchmarks; it scored 70.13% on Qwen3-30B-A3B, against an 81.83% no-memory baseline.
None of these vendors are lying. They are measuring a different thing. LongMemEval — 500 questions across information extraction, multi-session reasoning, temporal reasoning, knowledge updates and abstention — asks whether the system can find the fact. LoCoMo asks the same. Neither asks what finding it does to the reasoning that follows. Mem0 publishes 92.5 on LoCoMo at roughly 6,900 mean tokens per retrieval call against 25,000-plus for full context, with median latency flat at +1ms. Those are real engineering wins on retrieval cost. They are silent on this axis.
We have covered the upside of this architecture more than once — shared memory cutting token costs by 40% and Engram's $98M bet on an enterprise memory layer are both fair readings of the retrieval evidence, and this does not cancel them. Nor is it the first evidence pointing the other way. MemDelta varied one component at a time on LongMemEval-S in June 2026 and found agent self-memory at 42% against 47% for plain retrieval, with Mem0 matching cloud RAG on two of six question types at 50x the cost. AMA-Bench found memory systems generally failing to beat long-context baselines on agentic tasks. What MemTrapBench adds is the axis those two still do not score: not whether memory retrieves well, but what a correct retrieval does to the reasoning that follows.
This Is Not a Retrieval Bug, and That Is Why It Is Awkward
The paper's ablation kills the obvious explanation. Feed the model semantically relevant memory with the traps removed and Task Boundary accuracy hits 94.39% — above the 92.29% no-memory baseline in the same table. Feed it the trap memory and the same subset collapses to 31.05%. Same retrieval machinery, same relevance, opposite result.
So there is no index to fix, no chunking strategy to tune, no reranker to swap. The failure lives in what the model does with a correct fact, which is the same class of problem as Nvidia's cross-model KV cache transfer collapsing GSM8K accuracy — reused state that is technically valid and semantically wrong for the new job. It is also the same shape as a supervisor agent rewriting a task until the "do not" fell off.
None of this is unprecedented. The field has known since LLMs Get Lost In Multi-Turn Conversation — 200,000-plus simulated conversations across 15 models, an average 39% drop from single-turn to multi-turn, named an outstanding paper at ICLR 2026 — that accumulated conversational context degrades reliability, and that models which take a wrong turn do not recover. Memory frameworks were built to solve the long-conversation problem. The plausible reading of MemTrapBench is that they inherited it instead, and then made it durable across sessions.
Steel-man it properly, because the authors do. MemTrapBench is adversarial by construction — every instance is engineered so a prior misfires, which makes it, in the paper's own framing, a diagnostic stress test of harmful memory influence rather than a general evaluation of memory utility. It is 1,050 synthetic dialogues, two models, LLM-judged. One interest worth stating plainly, because the paper does not: four of the nine authors — including the first author and the senior author, both at Zhejiang University — also wrote LightMem, one of the five systems under test. It placed second on both models and posts the single largest gain under their own mitigation. Nothing in the results table depends on that, and the two lowest scorers are other people's systems, but it is the disclosure any vendor would be asked for. Real user histories are not 100% traps. Nobody should read this as "memory is bad." Read it as: the trap rate in your corpus is currently unmeasured, and the tooling assumed it was zero.
The Cheapest Mitigation Is a Paragraph, Not an Architecture
The paper's fix, AdaptiveMem, is a prompt. It is an inference-time instruction telling the model to check for memory traps before using memory — specifically to recognise when a task has shifted boundaries, avoid domain-locking to earlier turns, distinguish context-specific restrictions from universal rules, and separate role-play or counterfactual premises from real-world safety judgements.
On Gemini-3-Flash-Preview it recovered 14.9 points for LightMem, 11.8 for FullText and 11.3 for EverMemOS. On Qwen3-30B-A3B-Instruct-2507 the same instruction bought 2.5, 4.2 and 2.6 points — an order of magnitude less, which is itself the finding: prompt-level mitigation is model-dependent and you cannot assume the number transfers. On LongMemEval it improved four of six settings and left the other two unchanged, so it does not appear to cost you retrieval performance to run it.
Double-digit recovery from a system-prompt paragraph is the best cost-per-point available anywhere in this stack. Test it before you scope a memory-scoping project.
The architectural version already ships. Anthropic gives Claude a separate memory per project — "your product launch planning stays separate from client work" — which is a structural answer to the Task Boundary trap, the single worst category in the benchmark. Memory there is optional, must be enabled, and admins can disable it org-wide.
Your Assistant Vendor May Have Already Decided This For You
Check the default before you plan the eval, because at least one major vendor ships memory on. Microsoft's own admin documentation for Copilot personalization and memory — in preview, and updated 18 August 2026 — states it plainly: "No action is required to turn on Copilot memory; you only need to turn off Copilot memory for end-users or the tenant if desired." Enhanced personalization — the control that enables memory — is "by default, turned on."
Four operational facts from that same page belong in your risk register, not just your eval plan:
- Purview retention policies do not apply to Copilot memory. A "delete after three months" policy on Copilot Chat interactions does not delete the memories derived from it, and "there are no admin controls to enforce retention rules specifically for Copilot memory."
- Admins cannot restrict what type of information is added to memory.
- Memory and personalization actions generate no audit log entries in Purview.
- Memories live in the user's Exchange mailbox in a hidden folder, discoverable via eDiscovery under item class
IPM.Contactin theCopilotMemoryfolder.
That last one is the useful handle — it is how you find and purge, and it is the same mailbox-level persistence that made Copilot memory survive a credential reset a security problem earlier this month. Now it is also an accuracy problem: memory you cannot inspect, cannot scope and cannot expire is memory whose trap rate you cannot measure.
What to Do About It
This Week:
- Find the default. For Microsoft 365 Copilot, check the enhanced personalization setting via Microsoft Graph — it is on unless someone turned it off. Do the same for every assistant and agent framework in production. Write the answer down; most teams do not know it.
- Add memory-off as an arm in whatever eval you already run. Not a new eval — the same golden set, run twice. If memory-on does not beat memory-off on your traffic, you have bought latency and storage for a regression.
- Paste an AdaptiveMem-style scoping paragraph into one agent's system prompt and re-run. Four instructions, one afternoon, and on the paper's Gemini results a double-digit recovery.
This Month:
- Build twenty task-boundary cases from your own transcripts: a long session in one domain, then an unrelated question in the same thread. This is the trap that fell 40 points and it is trivially reproducible from real logs.
- Scope memory to the project or the task, not the user. Claude's per-project isolation is the reference implementation; if your framework only offers per-user memory, that is a gap to raise with the vendor now.
- Turn on trace-level visibility so you can see which memories were injected into a bad answer. Langfuse or equivalent — the agent monitoring stack we assessed earlier this month applies unchanged. Without the injected-memory record, a memory regression is indistinguishable from a model regression.
Before You Sign the Memory-Store Contract:
- Ask the vendor for MemTrapBench numbers, or for their own no-memory control. "State of the art on LoCoMo" now answers a question you are not asking.
- Put the control arm in the acceptance criteria, the same way you would grade a proof of concept pass/fail instead of on an average score.
- Pin the model. These effects are strongly model-dependent — 14.9 points of recovery on one model, 2.5 on another — so a silent model swap invalidates the result, exactly as a post-training update invalidated agent evals earlier this month.
The Bottom Line
Every enterprise AI feature of the last three years has arrived with an implicit claim that more context is strictly better: bigger windows, better retrieval, richer history. RAG made that claim and mostly delivered. Persistent memory makes it and, on this evidence, sometimes does the opposite — not because the plumbing fails, but because a correct fact from the wrong task is still the wrong input.
The industry ran this exact play with recommendation systems fifteen years ago. Personalization improved engagement metrics and quietly narrowed what anyone was shown, and it took years to name the failure because nobody was measuring the thing that broke. The measurement is the whole job. It always was.
You do not need to turn memory off. You need to be able to prove it helps.
Continue Reading
- Your Supervisor Rewrote the Task. The 'Do Not' Fell Off.
- Agents Averaged 73. Only 30% Were Usable. Grade Pass/Fail.
- Copilot Memory Survives Your Password Reset. Go Purge It.
- Nvidia Skipped the Prefill. The Math Went With It.
- Walmart and Amazon Both Said 40%. Neither Ran a Holdout.
- Best AI Agent Monitoring: Langfuse, Then a Real Kill Switch
- DeepSeek Swapped the Model. Your Eval Didn't Notice.
- How Shared Memory Cuts AI Token Costs 40% for Enterprises
