The permission allowlist your platform team checked into .claude/settings.json is not a permission allowlist. An entry reading Bash(python:*) looks like it says "the agent may run Python." What it actually authorizes is python -c followed by any string — which is every command on the machine, approved once, silently, for everyone who clones the repository.
That is not a thought experiment. A scan of 3,171 public GitHub repositories, submitted to arXiv on 7 September 2026, found that 16.0% of AI coding-agent setups carry at least one confirmed security defect in their configuration. Earlier work measured skill marketplaces and context files one component type at a time; this study audits the installed configuration as a whole, across component types and assistant platforms — and all three of its security classes land precisely on the controls engineering leaders believe are doing the protecting: permission grants that read as scoped and are not, skills that carry a shell approval into whoever installs them, and, the largest class at 9.8%, MCP server declarations that fetch and execute whatever a public registry happens to serve that morning.
What the Scanner Was Actually Looking At
The study, by four Red Hat researchers — worth noting, since the scanner they release is the one this article tells you to run — measures the agent harness, not the code the agent writes. An agent harness is the configuration layer — context files like CLAUDE.md and AGENTS.md, skills, slash commands, lifecycle hooks, MCP server declarations, subagent definitions and settings files — that a coding agent loads at session start and treats as instructions. Your CI scans the application. It almost certainly does not scan this.
The authors started from 9,295 candidate repositories and cut them down to 3,171, excluding 3,065 with no harness component at all, 1,633 carrying only an instruction file, 1,322 with a single component type and fewer than five skills, 65 forks, 22 failed clones and 17 that failed on rescan. What survived splits into two populations: 2,660 "setups", defined as a repository assembling at least two distinct component types — a context file and skills, skills and hooks, an MCP declaration and settings — and 511 "collections", meaning five or more skills with nothing composing them. The median setup carries 6 components and 7,057 tokens of instructions.
By client, Claude Code dominates the corpus at 1,887 repositories, followed by GitHub Copilot at 324, Cursor at 315, Gemini CLI at 273, Codex CLI at 212 and OpenCode at 142.
The headline number survives a serious validation pass, which is the reason to take it seriously at all. The raw instrument flagged 25.5% of setups. A second independent implementation re-derived every finding of every measured rule — 8,547 findings across 1,115 re-cloned repositories, not a sample — and disagreements went to a model adjudicator with the evidence a human reviewer would look at. That process cut the figure to 18.4% carrying any confirmed finding, 16.7% carrying a confirmed defect, and 16.0% carrying a security defect specifically: a 7.1-percentage-point false-positive reduction, published rather than buried.
Your Interpreter Grant Is a Shell Grant
An allowlist entry naming an interpreter is an allowlist entry naming every program on the host. The paper found that 3.1% of setups — 83 repositories carrying 161 grant findings — pre-approve arbitrary command execution through a permissions.allow entry that does not look like it does. The example forms it names are Bash(awk:*), Bash(python:*), Bash(find:*) and Bash(sed:*). Each of those programs has a documented escape hatch: awk has system(), Python has -c, find has -exec, and GNU sed has the e flag.
The 83 break down into shell-escape tools (58 setups), interpreter access (54), unrestricted shell (19), package runners such as npx, bunx, uvx and pipx (18), and the shell invoked by name (10). Setups count once per mechanism, which is why the mechanisms sum higher than the repository count.
The vendor documentation does not dispute any of this. Claude Code's own permissions reference states that a Bash rule "doesn't match the same program invoked in a different form, so a deny or ask rule covers the invocation Claude usually produces and isn't a security boundary around the program." The same page warns that in Bash(git * main) the wildcard covers -c, "which makes git run a program you name," and that environment runners including npx and docker exec are not stripped wrappers — so a rule like Bash(devbox run *) "matches whatever comes after run, including devbox run rm -rf .". The behaviour is documented. It is the interface that misrepresents it, which is why the paper's recommendation to Anthropic is a permission UI that renders Bash(python:*) as "any command" at the moment somebody approves it.
Here is the steel-man, and it is a fair one: 83 repositories is a small number, and almost none of those grants were written by an attacker. They were written by the permission dialog, when a developer hit "yes, and don't ask again" on a perfectly reasonable python scripts/build.py. That is exactly the problem. The dangerous rule is a byproduct of a convenience flow, nobody typed it deliberately, and nobody reviews it afterwards — which is the same failure mode as Claude Code's move to auto-approval by default, where deny rules and not prompts became the only real policy surface.
Nor are these grants inert. Of the 65 setups whose grant falls in the security class, all re-cloned at their pinned commits, 22 ship a component that invokes the granted interpreter or tool in an instruction context. The permission and the instruction that uses it are in the same repository.
npx -y Is a Standing Order to Run Today's Code
9.8% of setups declare at least one MCP server whose package carries no version — the single largest defect class in the study, and the one with the clearest supply-chain shape. The typical form the authors found is npx -y @scope/server-name, with uvx and docker without a tag or digest appearing in the same role. Their description of the consequence is exact: the server "runs with the agent's privileges, is fetched from a public registry on each start, and executes whatever the registry serves that day."
The -y is not incidental. npm's own documentation explains the flag's purpose without euphemism: "To prevent security and user-experience problems from mistyping package names, npx prompts before installing anything. Suppress this prompt with the -y or --yes option." The confirmation exists because of this risk class. The canonical MCP declaration turns it off.
This is reachable, not theoretical. In a seeded sample of 40 setups with an unpinned server, 29 ship a skill, command or context file that names that server or one of its tools — the agent is being told to use it.
And it has already happened once in public. In September 2025 a package called postmark-mcp appeared on npm, built trust across 15 versions, then added a backdoor in version 1.0.16 that silently BCC'd every outgoing email to an attacker-controlled address. It was downloaded 1,643 times before removal, and the malicious change was a single line in an otherwise faithful clone of a legitimate library. Postmark's statement was blunt: the company "had absolutely nothing to do with this package." Every configuration that read npx -y postmark-mcp picked up 1.0.16 at the next session start. No prompt, no pull request, no diff, nothing in the audit trail — the same persistence property that made the ChainDrop worm survive npm's own takedown by living in agent config.
The fix the authors propose is the one the packaging world settled on twenty years ago: "Server declarations need the equivalent of a lockfile. The minimum is a client that warns when npx -y carries no version; the right answer is a resolved manifest with digests, generated on first run and committed beside the configuration."
What exists today is narrower than that. GitHub gives enterprise administrators an MCP server allowlist in managed-settings.json, described as "the more secure, generally available method for restricting access to MCP servers," plus a custom-registry option still in public preview. That governs which servers your developers may run. It does not govern which version, which is the gap this study measures — the same distinction that made artifact pinning rather than vendor trust the right answer for model weights.
The Skill That Installs a Shell Pre-Approval
A skill can carry its own permission grant, and installing the skill installs the grant. 3.8% of setups and 3.7% of collections ship a skill whose allowed-tools list includes an unrestricted Bash or a shell-escape command — 121 repositories, every one of which was independently re-derived and confirmed, with no adjudication required.
The allowed-tools field in a skill's frontmatter declares the tools the client runs without prompting while that skill is active. It is a useful feature and a portable one, which is the trouble: the pre-approval travels with the artifact into whatever repository imports it. A developer installing a well-reviewed skill from a marketplace is also installing a standing shell approval they never saw.
The authors single this one out for a structural reason worth repeating to whoever runs your internal marketplace: "This is the one security class that appears at publication time rather than assembly time, and it is the one a marketplace scan can see." Every other defect in the study is assembled locally and can only be caught locally. This one can be caught once, centrally, for everybody — and currently is not.
A smaller finding in the same family did not clear the paper's reporting bar but belongs in your grep: 0.4% of setups, 12 findings in total, commit permissions.defaultMode set to bypassPermissions or dontAsk in project settings, which removed the permission prompt for every user who opened that repository. Claude Code has ignored bypassPermissions from project scope since v2.1.257. The lines are still committed, and still take effect on older clients — which is precisely why agent configuration deserves a stricter review tier than documentation, a point that surfaced when only 16% of commits labelled "docs" in Claude Code plugin repos actually changed documentation.
What This Study Does Not Show
The paper is unusually honest about its own limits, and reading them is what turns the number into a decision rather than a scare. Four constraints matter.
No compromise was found. The authors report no confirmed credential-exfiltration paths anywhere in the corpus. What they measured is capability — pre-approvals that are wider than they look and code paths that fetch unreviewed artifacts — not evidence that anyone exploited them.
The sample leans toward exhibitionists. Discovery ran through topic queries, curated lists and plugin marketplaces, so the corpus "over-represents repositories that advertise or distribute their agent tooling." Those repositories are more sophisticated than average, which cuts in both directions.
Every figure is a floor. In the authors' words: "We do not know what the 6 gating rules miss, and every headline figure is a lower bound on the defects the instrument can express." And no human ever scored the adjudicated cases — the second implementation was written by the same team, so "a shared misreading of a format would pass both."
Your repositories were not in it. This is the limitation that should move you: "The corpus is public GitHub; configurations maintained privately inside organizations may differ systematically and are the population most relevant to enterprise adoption." Nobody has measured your fleet. The 16.0% is a prior, not a finding about you — and the only way to convert it into a finding about you is to run the scan.
Run the Scanner Before Your Next Fleet Rollout
This Week: Install the instrument the authors released and point it at your ten busiest repositories. It is harness-eval, Apache 2.0, pip install harness-eval, then harness-eval harness-security . for the security scan. The full linter is harness-eval harness-lint . — 108 deterministic rules across 12 categories, and the command that takes --format sarif — while harness-eval harness-gate . runs only the 21 gating-tier rules and exits 1 on any finding, which is the form to block a build on. The paper's claim is modest and checkable: "the released instrument runs in seconds and emits SARIF, so it attaches to code scanning like any dependency check." In parallel, grep every settings.json in your estate for permissions.allow entries naming an interpreter or a package runner — python, node, ruby, perl, awk, sed, find, npx, bunx, uvx, pipx, docker, make, ssh — and count them. That grep takes an afternoon and it is the finding you will have to explain to your security lead anyway.
This Month: Pin every MCP server declaration. Replace npx -y @scope/server with an explicit version, or a digest-pinned container, and put the resolved list under source control next to the config. Then decide who may add one — the GitHub allowlist governs which servers, so the version pin has to live in your repository regardless of which client your developers use. And move agent configuration out of the documentation review tier: a change to .claude/settings.json, .mcp.json or a skill's frontmatter is a change to executable policy, so put CODEOWNERS on those paths and require a security reviewer, exactly as you would for a Terraform IAM module.
Before Your Next Fleet Rollout: Stop treating the allowlist as the boundary. Claude Code's documentation says where the boundary actually is — for "filesystem and network enforcement that doesn't depend on the command text," you want sandboxing and a PreToolUse hook that inspects the full command string, not a prefix rule. The allowlist is a prompt-fatigue tool. A container with an egress policy and short-lived scoped credentials is what contains the command that slips past it, and the difference showed up the hard way when a Cursor agent that refused a task in one session complied after a restart with the same credentials still attached. If you want to reproduce or extend the numbers, the full study artifact ships the rule catalog, the corpus manifest with pinned commits, every finding, the adjudication verdicts and the scripts that regenerate each figure.
The Bottom Line
Software has run this exact play before. Between roughly 2010 and 2016 the npm ecosystem learned that a version range in a manifest is a standing promise to execute code nobody on your team has read, and the industry answered with lockfiles, integrity hashes, provenance attestation and a registry you could audit after the fact. None of that arrived because the risk was newly discovered. It arrived because somebody finally counted how often the loose form was in use, and the number was embarrassing.
Agent configuration is at the pre-lockfile stage of that same arc. The declaration format has no version discipline, the canonical incantation carries a flag whose documented job is to suppress a security prompt, and the permission vocabulary renders "any command on this machine" as a tidy parenthesised program name. Every one of those is fixable, and the paper assigns each fix an owner: clients and the MCP spec for pinning, Claude Code for the permission UI, marketplaces for allowed-tools disclosure at install time. None of those owners will ship in time for your next quarter.
You get the scan and the grep. Do them.
The allowlist is not the boundary. It is the list of things you stopped watching.
Continue Reading
- Claude Code Stops Asking Aug 14. Prompts Aren't Policy.
- npm Pulled the Packages. Your Agent Config Reinfects You.
- Only 16% of 'Docs' Commits Were Docs. Re-Tier Review.
- Guardrails' Hub Died Aug 25. Harvey Bought the Team.
- Antigravity's Allowlist Isn't Honored. Use Your Proxy.
- What Is an Agent Harness? The $1B Claude Code Architecture
- 12 Models Wrote a Fake Crate. None Refused a Real One.
