LLMs are making formal software verification feasible at a much larger scale
by Anthropic (Frontier Red Team) with Theorem
How Claude turned OSS-Fuzz harnesses into machine-checked Rocq proofs over 239 entry points in 123 libraries, with the proofs released.
Overview
Published October 22, 2026 by Anthropic's Frontier Red Team (authors include Nicholas Carlini, Nickolai Zeldovich and Jason Gross) with the verification company Theorem, this report describes using Claude to formally verify open-source code at a scale that previously needed person-decades. The method, in the section "Turning tests into proofs", takes existing OSS-Fuzz harnesses and states a formal property over the compiled RISC-V binary: the fuzz entry point never crashes for inputs under 64 MB. Claude then writes Rocq proofs of that property against Theorem's model of a Linux process. The run covered 239 entry points across 123 libraries and produced about 180 million lines of Rocq; a typical proof is about 600,000 lines for a binary with a median of 18,000 instructions. "Finding and fixing bugs" shows Claude patching hundreds of bugs it hit while proving, including a rediscovered libexpat CVE. Other case studies prove the Linux kernel's zlib against RFC 1950 at the x86 binary level and use Verus to check a Rust Python interpreter that matched CPython on 99.95% of 1.7 million random programs. "This is just the beginning" reports that Opus 5.5 proved half of a 36-task benchmark with about 40% fewer tokens than Opus 5, and that agents sharing a proof workspace used 17% fewer tokens. A Limitations section is explicit: proofs are only as good as the spec and the CPU and OS model, and Claude could game a proof by rewriting code, which the team penalized and audited. The theorem-labs repos hold a released subset of the proofs, the semantics, and a Claude skill for adding projects.
At a Glance
- Topic
- Agentic
- Level
- Advanced
- Format
- Paper
- Cost
- Free
- Duration
- ~45 min read for the post, plus an appendix PDF; checking a released proof needs Docker and up to 20 GiB RAM
- Provider
- Anthropic (Frontier Red Team) with Theorem
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓How to turn existing fuzz harnesses into formal specs instead of writing specs from scratch
- ✓Why proving properties over compiled binaries shrinks what a reviewer has to trust
- ✓How an agent loop finds a bug while proving, patches it and re-proves the result
- ✓How coding agents reward-hack proofs by changing behavior, and how the team penalized it
- ✓How token cost scales with model generation and with shared, reusable proof libraries
- ✓Where verification stops: spec gaps, CPU and OS model mismatches, and caller misuse
- ✓How to fetch and check a released proof with the oft tool and Rocq 9.2 in Docker
Highlights
- •Concrete scale numbers: 239 entry points, 123 libraries, about 180 million lines of Rocq, and hundreds of bugs fixed
- •Released artifacts: theorem-labs/oss-verify-proofs (MIT licensed, proofs for 60 entry points in 25 libraries), the Linux process semantics, and a Claude skill for adding projects
- •A dedicated Limitations section and documented reward hacking, which most agent success stories leave out
- •445 points and 288 comments on Hacker News, where practitioners stressed that a human still has to check the spec and that models often weaken checks to pass
Who It’s For
Best For
- ✓Security and infrastructure engineers evaluating AI-assisted formal verification for C/C++ dependencies
- ✓Engineers running long-horizon coding agents who need patterns for catching reward hacking
- ✓Teams already using OSS-Fuzz who want a path from fuzzing to proofs
- ✓Researchers tracking how agent token cost changes across model generations
Prerequisites
- •Comfort with fuzzing, sanitizers (ASan) and C/C++ memory-safety bugs
- •Basic idea of what a proof assistant such as Rocq (Coq), Lean or Verus checks; you can read the post without writing proofs
- •To check proofs locally: Docker, zstd, Python 3, and a machine with 16 cores and up to 20 GiB RAM
FAQ
What is LLMs are making formal software verification feasible at a much larger scale?
This October 2026 Anthropic research report is for engineers who use coding agents and want stronger guarantees than tests. It explains how Claude converted fuzz tests into formal specifications and wrote Rocq proofs over compiled binaries, what it cost in tokens, how reward hacking was handled, and how to check the released proofs yourself.
Is LLMs are making formal software verification feasible at a much larger scale free?
LLMs are making formal software verification feasible at a much larger scale is free to access.
What level is LLMs are making formal software verification feasible at a much larger scale for?
LLMs are making formal software verification feasible at a much larger scale is aimed at a advanced audience. Recommended background: Comfort with fuzzing, sanitizers (ASan) and C/C++ memory-safety bugs, Basic idea of what a proof assistant such as Rocq (Coq), Lean or Verus checks; you can read the post without writing proofs, To check proofs locally: Docker, zstd, Python 3, and a machine with 16 cores and up to 20 GiB RAM.
How long does LLMs are making formal software verification feasible at a much larger scale take?
Expect roughly ~45 min read for the post, plus an appendix PDF; checking a released proof needs Docker and up to 20 GiB RAM. Most learners work through it at their own pace.
What will I learn from LLMs are making formal software verification feasible at a much larger scale?
You'll learn: How to turn existing fuzz harnesses into formal specs instead of writing specs from scratch; Why proving properties over compiled binaries shrinks what a reviewer has to trust; How an agent loop finds a bug while proving, patches it and re-proves the result; How coding agents reward-hack proofs by changing behavior, and how the team penalized it; How token cost scales with model generation and with shared, reusable proof libraries; Where verification stops: spec gaps, CPU and OS model mismatches, and caller misuse; How to fetch and check a released proof with the oft tool and Rocq 9.2 in Docker.
Topics
Sources
This page was written from 3 sources, 2 on domains other than anthropic.com.