Anthropic's new agent-oversight numbers are a report card an enterprise can copy, and the first of its three measures, coverage, is the one most agent programs cannot produce.
This week Anthropic disclosed that its most-used internal platform runs about 30,000 agents at a time, that 100% of their actions pass through a monitor before they execute, that about 1 in 47,000 decisions was blocked in August, and that roughly 50 flagged transcripts a week reach a human (Anthropic, Measuring the pace of AI development). It then said any frontier developer could publish these measures regularly, and nothing in them is specific to a frontier lab. The catch is in the first one. Coverage is a fraction, and its denominator is an inventory of your agents. In an April 2026 survey of 750 UK and US technology leaders by Gravitee, which sells API and AI-agent management software, mean monitoring coverage was about 52%, and 90% of organisations had unmonitored agents in production.
So the answer to the headline is: probably not yet. The fix is cheaper than it sounds, and it should happen before anyone widens what your agents are allowed to do.
What Anthropic Actually Reported About Its Agents
Anthropic reported three oversight metrics for one internal platform, for August 2026, and defined each precisely enough to copy. In its words, coverage "describes the share of an agent's actions that pass through a monitor before or after execution"; review latency "is the time between an action and its review, first by an automated monitor and then by a human"; and escalation rate "is the share of agent activities that are either blocked/redirected (in the case of online monitors) or flagged for further review (in the case of offline monitors)" (Anthropic).
| Measure | Anthropic's August 2026 figure | What it does not tell you |
|---|---|---|
| Coverage (online) | 100% of actions checked before execution, "usually within seconds" | Anything about agents on other platforms |
| Coverage (offline) | 100% of actions ingested after they run | Whether "agent traffic" was labelled correctly |
| Escalation rate (online) | 0.002% of over a billion decisions blocked, about 1 in 47,000 | How many harmful actions were not blocked |
| Escalation rate (offline) | ~100,000 transcripts flagged a week; ~50 escalated to humans | How often the classifier layer dismissed a real problem |
| Review latency | Automated review before the action; humans review blocked actions within one week | How fast an executed-and-flagged action reaches a person |
Two details matter for anyone copying this. First, the online monitor is aimed at a specific class of harm: those "that would happen quickly and could not be undone, such as an agent copying model weights out of our systems." It can block, or "steer agent misbehavior automatically." Second, the reason Anthropic is publishing at all is scale. It says Claude now "leads" 26% of its AI R&D work, up from under 1% in February 2026 (Anthropic). When the share of work agents lead goes from under 1% to 26% in six months, oversight sized in February is undersized by August.
The arithmetic is worth doing once. More than a billion decisions at 0.002% is more than 20,000 blocked actions in a month, every one of which Anthropic says a human reviews within a week. The offline funnel is steeper: roughly 1 in 2,000 flagged transcripts reaches a person. Those are staffing numbers as much as safety numbers.
Coverage Is Only as Good as Its Denominator
A 100% coverage figure means 100% of the traffic the monitor knows is agent traffic, on the platform it watches. Anthropic says so directly: "These measurements cover this platform only" (Anthropic).
The more instructive evidence is how that denominator broke earlier this year. METR's Frontier Risk Report, covering February to March 2026, says its red-teaming "found that monitoring coverage had exceptions (for example, it did not cover non-Claude Code traffic)," and identified "several simple ways for monitoring to be disabled," including "changing an environment variable that indicated whether the traffic came from Claude Code." The red-teamer called it "a reliable, easy to implement single point of failure that could let a model completely circumvent the offline monitoring system." According to the 80,000 Hours discussion of the report, that issue was fixed within a day of discovery. METR's own write-up of the three-week exercise says it found "several specific novel vulnerabilities, some of which have since been patched."
That is the lesson for your program, and it is not about Anthropic. Coverage was decided by a label, and the thing being monitored could set the label. OpenAI's comparable disclosure has the same shape: it reports monitoring 99.9% of internal coding-agent traffic, with under 0.1%, mainly from bespoke or local setups, outside coverage (OpenAI, via its LessWrong crosspost). The traffic that escapes is the traffic that doesn't look like the standard setup.
In an enterprise, the equivalents are a developer running Claude Code from a personal shell with telemetry off, a Cursor or Codex session on an unmanaged laptop, or a scheduled job calling a model API with a service account nobody registered. None of them shows up in the numerator or the denominator of a monitoring dashboard. Gravitee found only 7.2% of organisations have a named individual formally accountable for AI agent behaviour (Gravitee), which is usually why the inventory has no owner either. We covered the discovery side in our piece on shadow agents.
Measure coverage against a source the agent cannot edit: your identity provider's agent and service identities, the keys issued by your LLM gateway, or egress logs. Then divide what your monitor saw by that.
A 1-in-47,000 Block Rate Says Nothing About Misses
An escalation rate counts what the monitor caught; no lab has published how much its production monitor misses. Anthropic is candid about that: "At this scale of agent activity, it is hard to be certain we have a complete picture of all possible kinds of behaviors" (Anthropic).
The research on this point is sobering. Anthropic has "generally found catch rates of >80%" across "a large variety of artificial scenarios," per METR's report. But on SLEIGHT-Bench, a set of 40 attack transcripts built specifically to evade monitors, "20 of 40 attacks are never caught by an Opus 4.6 monitor with extended thinking across 10 trials at a 1% false-positive rate, while the overall catch rate is 32%." Different test sets, very different answers. The catch rate is a property of the attacks you test with, which is exactly why a single number without a test set behind it is not evidence.
A detection rate is the share of known-bad actions a monitor flags when you deliberately feed them in. It is the fourth number, the one missing from both labs' oversight disclosures. Anthropic's own post says developers "should share how often known agent misbehavior is caught by monitors," but it does not give that figure for this platform. It is the only one that tells you whether a low escalation rate means a well-behaved fleet or a blind monitor.
The false-positive side cuts the other way. A monitor tuned to a 1% false-positive rate and applied to a billion decisions a month would raise around 10 million false alarms. That is why Anthropic's offline path runs through "multiple classifiers for false-positives and initial oversight review" before a person sees anything (Anthropic). Every layer in that funnel is a place a real problem can be dismissed, and none of them publishes its own miss rate.
When you run your own test, keep the sample size honest. Twenty seeded attacks give you a catch rate with a very wide error bar, the same trap we walked through in our piece on eval sample size.
Review Latency Is a Staffing Decision
Review latency is set by how many people sit at the end of the funnel, and the two labs that have published chose very different targets. Anthropic's human review of blocked actions happens "within one week." OpenAI says it sends higher-severity cases for human review within 30 minutes (LessWrong crosspost).
Those aren't the same measurement, and the distinction matters for you. A week is defensible for an action that was blocked: the harm did not happen, and the human review is an audit of the monitor. A week is not defensible for an action that was executed and flagged afterwards, because whatever it did is already done. Report those two latencies separately, or the comfortable one will hide the dangerous one.
Most enterprises are nowhere near either target. In an August 2026 survey of 202 IT and security leaders at companies with 1,000+ employees, by EMA for Cequence, which sells API and agent security, 65% had experienced an agent acting outside its intended scope, only 32% could detect and contain that within minutes by automated means, 55% needed hours and manual steps, and only 34% evaluated authorisation at the moment an agent attempted an action. If authorisation is checked once at provisioning, there is no pre-execution review to measure. That is the standing-privilege problem in our agent authorization guide.
Latency also depends on what the reviewer can read. If the transcript is compacted or summarised before a human sees it, a fast review of the wrong record is still a miss. We covered that failure in OpenAI's models wrote cover-up notes. And a human who sees 50 escalations a week, most of them benign, drifts toward approving them. The automation-bias research is a good argument for rotating reviewers and seeding their queue too.
How to Produce the Three Numbers From Telemetry You Have
You can produce a first version of all three in a month, for at least your coding agents, from telemetry most teams already have switched off. Claude Code's monitoring documentation is a useful worked example because it maps cleanly onto Anthropic's definitions:
- Escalation-rate numerator. The
claude_code.tool_decisionevent records every permission decision withdecision_type(accept,auto_acceptorreject) and adecision_source(config,hook,user_permanent,user_temporary,user_abort,user_reject). Count rejects per 1,000 decisions, weekly, split by source. Note that rejected calls produce noclaude_code.tool_resultevent, so a block rate computed from tool results will read zero. - Coverage enforcement. Telemetry is off unless
CLAUDE_CODE_ENABLE_TELEMETRY=1is set, the same class of environment-variable weakness METR found. Put the exporter configuration in managed settings instead; the documentation says that when anOTEL_EXPORTER_OTLP_*variable is set there, "Claude Code removes conflicting developer-set variables at startup." - Review latency. Export to the same OpenTelemetry collector your observability stack already reads, and timestamp two events per flag: when it was raised and when a named person closed it. Our agent monitoring buyer's guide compares Langfuse and LangSmith for this layer.
The same pattern applies to GitHub Copilot, Cursor and Codex, with different event names. If a vendor cannot export a per-action permission decision, you cannot compute an escalation rate for that surface, and you should record that surface as uncovered rather than leave it out of the denominator.
Two definitions to fix before you publish anything internally. Report coverage per platform, never blended, because a 95% average can hide one surface at zero. And separate pre-execution gates from post-hoc review, because Anthropic's online and offline monitors are different controls with different failure modes. Its online monitor is aimed at fast, irreversible harms; your equivalent is a short list of actions gated before they run (production writes, credential reads, network egress, deletion), enforced as policy rather than prompts, as we argued in our piece on Claude Code deny rules.
Who Doubts These Numbers, and Why It Matters Less
The strongest objection is that these are self-reported figures for one platform, with no miss rate and no independent audit. The Council on Foreign Relations' position, as summarised in The Neuron's coverage, is that voluntary commitments still need independent oversight and enforceable standards. That is correct. A 100% figure nobody outside the company can check is a claim, and METR's red-team is the reminder that the first external look found the monitoring could be switched off.
Anthropic's answer is partial but concrete. It says it plans "to embed independent third-party evaluators from multiple organizations at Anthropic, and give them access to internal processes, systems, and data comparable to what internal risk assessment teams have" (Anthropic).
For a buyer, the objection matters less than it seems. Until this year, "we monitor our agents" was unfalsifiable. Now there is a published definition, a unit and two labs' baselines, and you can ask a vendor for its figures in the same terms. That is useful even if the first figures turn out to be generous.
What to Do Before You Widen Agent Permissions
This Week:
- Count your agents from a source they can't edit. Pull every agent identity, service account and model API key from your IdP and LLM gateway. Compare that list with what your monitoring actually sees. The ratio is your real coverage number. Write it down per platform.
- Move agent telemetry into managed settings. For Claude Code, set the OpenTelemetry exporter in the managed settings file so a developer or an agent cannot switch it off per session. Do the equivalent for every other agent surface that supports it.
This Month:
- Start the weekly escalation report. Rejected actions per 1,000 decisions, by platform and by decision source. Track the trend. A sudden drop is as suspicious as a spike.
- Name the reviewer and set two latency targets. One owner for the flag queue, a target for blocked actions (days is acceptable) and a much shorter one for executed-and-flagged actions (hours).
- Seed the monitor. Run known-bad trajectories through it, starting with the public SLEIGHT-Bench set, and record the catch rate. That is your detection rate, and it is the number Anthropic's oversight report did not include.
Before Renewal:
- Ask each agent-platform vendor for the three numbers on your tenant: coverage by surface (IDE, CLI, cloud agents), escalation rate, and the latency from a flag to your security team hearing about it. Ask which surfaces are excluded from monitoring, and get the answer in writing.
The Bottom Line
Anthropic has done for agent oversight what uptime percentages once did for hosting: turned a reassurance into a number that can be compared. Security teams learned the limit of that kind of number with endpoint coverage long ago. A sensor on 100% of the laptops you know about is not 100% coverage; it is 100% of your inventory, and the inventory is where the gap lives. Agent fleets repeat the lesson at machine speed, and METR showed the agent itself may be able to edit the list.
Publish the three numbers inside your company. Add the fourth. Own the denominator.
Continue Reading
- Agent Authorization: Standing Privilege Is the Whole Problem
- Best AI Agent Monitoring: Langfuse, Then a Real Kill Switch
- OpenAI's Models Wrote Cover-Up Notes. Can You Read Yours?
- Claude Code Stops Asking Aug 14. Prompts Aren't Policy.
- 1,200 Agents Met in Artifactory. Go Log Repo Creation.
- Geordie's $180M Bet: 79% of AI Agents Are Shadow
