Your AI review bot probably works for the same company that wrote the code it is reviewing. A new measurement of public GitHub found 248,641 agent-authored pull requests that received at least one AI-attributed review — and 208,145 of them were reviewed by the same product that authored them, against 45,269 reviewed by a different one. Roughly five in six closed-loop reviews are the vendor checking its own work.
That is not a scandal. It is a default. Review bots get installed alongside the authoring agent, and nobody chose it. But if your branch protection treats an agent's pass as a review gate, you are counting one vendor twice — and the fix is a one-line merge policy: the reviewing agent must be a different product than the authoring agent.
What the Study Actually Counted
The paper measured who reviewed whom, not whether the reviews were good. "AI-to-AI Code Reviews of GitHub Pull Requests," by Niruthiha Selvanayagam and Taher A. Ghaleb, was posted to arXiv on 21 August 2026 and accepted at ESEM 2026's Emerging Results track. It links AI-attributed PRs to AI-attributed review events in CodAGE, a public dataset of coding-agent GitHub events covering 1 January 2024 to 15 April 2026.
The pipeline filtered 4,563,819 candidate PRs and quarantined 1,733,535 of them — 38.0% — for lacking sufficient attribution evidence, retaining 2,830,284 agent-authored PRs. Of those, 248,641 (8.8%) drew at least one AI review. The split: 45,269 cross-product, 208,145 same-product, 4,773 that got both. The arithmetic closes exactly, which is more than most vendor telemetry manages.
Two caveats matter before you act on this, and the authors state both plainly. First, "closed-loop" here means AI occupied both sides of the PR, not that humans were absent — the dataset only observes AI review events, so a human may also have reviewed. Second, attribution depends on signatures like Co-Authored-By: Claude or a coderabbitai[bot] login, so every absolute count is a lower bound.
What is not in doubt is the trajectory. Closed-loop review went from 57 cross-product and 40 same-product PRs in 2025-Q1 to 25,492 and 57,080 in 2025-Q3 — more than two orders of magnitude in two quarters, with the cross-product half alone spanning 10,345 repositories.
Same-Product Review Is the Default Because the Install Is
The concentration is a distribution artifact, not a preference anyone expressed. The per-vendor numbers make that obvious: Copilot-authored PRs were 95.7% same-product reviewed across 173,957 pairs, and Amazon Q sat at 91.8%. Meanwhile Cursor-authored PRs were 100% cross-product and Claude Code-authored PRs 99.5%.
In other words, GitHub Copilot reviews Copilot's code because both arrive with the same installation. The authors say so themselves: same- and cross-product patterns "may reflect product design", differing in integration and triggering rather than in anyone's judgment about who should review what.
That is exactly why it is worth a policy. Nobody at your company decided that the author's vendor should also be the reviewer. It happened during an app install, and it now covers a large share of your agent-authored diffs. The most common cross-product pairing in the dataset is Codex-authored, Copilot-reviewed at 18,114 pairs — a genuinely independent combination that happened by accident too.
The Gate You Think You Have, You Probably Don't
Before you change policy, find out which of your bots can actually approve a pull request — because they do not behave the same way. This is the single most useful thing to check this week, and most teams have never checked it.
GitHub Copilot code review cannot gate anything. Per GitHub's own documentation, Copilot always leaves a "Comment" review, not an "Approve" or "Request changes" review, and "its reviews do not count toward required approvals and will not block merging". If your engineers believe Copilot's pass is a review gate, they are wrong in the safe direction.
CodeRabbit is different. @coderabbitai approve submits a real approval — but only when reviews.request_changes_workflow is enabled in .coderabbit.yaml; without it, CodeRabbit resolves its threads and reports that approval is disabled. That key defaults to false, so the question is whether somebody turned it on — and once it is on, CodeRabbit can approve automatically when its own comments are resolved and no pre-merge check is failing. That single YAML key is the difference between a bot that comments and a bot that satisfies a required-approval rule. Go read the file.
The irony is that GitHub already enforces reviewer independence — just on the human side. If a person kicks off a Copilot coding agent PR, their approval "won't count toward the required number" and another reviewer must approve it. Teams have complained that this blocks them. GitHub is right and they are wrong: collaborating on a change disqualifies you from independently approving it. That principle has simply never been extended to the agent doing the reviewing.
Independence Matters Even If the Bias Doesn't
Be careful here, because the tempting argument is the weak one. The obvious worry is self-preference — that a model goes easy on its own output. The evidence for that is real but general, not specific to code: LLM judges show a linear correlation between recognising their own text and preferring it, and a 2026 analysis of 20 models defines self-preference bias as a directional deviation in which models systematically favour or disfavour their own outputs. Nobody has demonstrated this for code review at scale, and this paper did not try. It measured concentration, not leniency.
The stronger argument does not need bias at all. It needs correlated failure. Two instances of the same model family, on the same context window, with the same training data and the same blind spots, are more likely to miss the same class of bug — and a reviewer that misses what the author missed is not a control, it is a copy. This is the same reasoning that made one assurance firm running three frontier labs' cyber evaluations a concentration problem rather than an efficiency win, and the same reason inheriting a vendor's own safety evaluations is not due diligence.
Steel-man the other side, because it is not stupid: cross-vendor review costs a second subscription, doubles the comment volume on every PR, and there is no published evidence that it catches more bugs. The sharpest form of the objection is that a review pass is stateless and runs on different scaffolding than the authoring session, so it is not self-evidently the same reader twice — Greptile makes exactly that case while disclosing that it sells an AI code reviewer, which is to say against its own commercial interest. And on the one public benchmark that tests aggregation, running a single model ten times raised review F1 by 43.67% — repetition alone recovers a good deal of what one pass misses, with no change of vendor. If your agent-authored PRs already get a real human approval, the marginal value of a second bot is close to zero. The policy below is aimed at the teams where the bot's pass is the last thing standing between a diff and production — which, given that a third of PRs at one large engineering org are now auto-approved, is more teams than admit it.
The Reviewer's Output Changed With the Author
Reviewer behaviour was not constant across authoring agents, though the effect is smaller than the headline number suggests. Across 35,248 CodeRabbit comments classified by the tool's own category headers, the refactor share ranged from 9.7% on Devin-authored PRs to 35.0% on Claude Code-authored PRs, against 10.5% on Copilot-authored PRs — a 25.3-point spread on the same reviewer.
Do not over-read it. The association is statistically overwhelming and practically small (Cramér's V = 0.150), the categories are the vendor's self-labels rather than validated severity, and the authors warn the difference may reflect the PRs rather than the reviewer. Devin and Copilot PRs are not Claude Code PRs.
The per-PR volume finding is similar in shape. Three of four dual-role reviewers produced 58–65% more comments in same-product configurations — Copilot 2.35 versus 1.49 comments per PR, Devin 1.80 versus 1.09, Amazon Q 8.08 versus 4.94 — but effect sizes were small to negligible, and OpenAI Codex showed none at all (0.89 versus 0.91). If you expected same-vendor reviewers to be visibly lazier, the data does not support it.
What the data does support is the thing every engineering leader already suspects: more comments is not more safety. Developers accepted 36.4% of CodeRabbit's suggestions, rejected 56.3%, and discussed 7.3% across 31,073 review-feedback pairs from 10,191 PRs in 239 repositories — with rejections driven by false positives and misalignment with project norms. A review stream your engineers reject the majority of is not a gate. It is a volume problem your triage layer absorbs.
Write the Merge Policy This Way
This Week:
- Enumerate every bot with write access that can submit an approving review. In each repo, open
.coderabbit.yamland check whetherreviews.request_changes_workflowistrue. That key decides whether your reviewer is advisory or load-bearing, and it is usually set by whoever installed the app. - Pull the last 200 merged PRs and count how many cleared required approvals with zero approving review from a human. This is a query, not a project. If the number is not zero, you have already shipped code that no person approved.
- Tell your engineers what Copilot's review is and is not. It is a comment. It never blocked anything. The belief that it did is more dangerous than the tool.
This Month:
- Write the rule in one sentence: an approving review from the same product family as the authoring agent does not satisfy the independent-review requirement. Then accept that GitHub cannot express it natively — the ruleset options run to required approvals, dismiss stale approvals, Code Owners, required reviewer teams, most-recent-push approval, thread resolution, and one agent-aware rule — an extra approval for Copilot pull requests not attributed to a person, on by default. That last one keys on who wrote the PR; none of them know what vendor a reviewer is. Enforce it with a required status check that reads the PR's authoring signature and fails when the approving app matches.
- Turn on "require an approval from someone other than the last person to push." It is the closest native analogue to independence and you almost certainly already have it available.
- Name the blast radius where cross-vendor review is mandatory — authentication, payments, schema migrations, infrastructure-as-code, anything touching customer data. Everywhere else, one bot is fine.
Before Renewal:
- Ask each vendor in writing whether their review product can submit an approving review, and how to turn that off. Get it in the contract or the admin console, not a support thread.
- Stop paying two subscriptions to one vendor for both sides of the PR. If you are buying an author and a reviewer, buy them from different companies — that is the whole point, and it costs nothing extra.
- Measure it like an experiment, not a belief. Route half your agent PRs to a cross-vendor reviewer for a quarter and compare escaped-defect rates. Nobody has run this test publicly, which is precisely why you should run a holdout instead of accepting a vendor's number.
The Bottom Line
Every previous generation of software assurance eventually learned the same lesson: the checker cannot report to the builder. Financial audit split from bookkeeping. Security testing split from development. QA got its own reporting line, lost it, and got it back. Each split was resisted on cost grounds and each one was later treated as obvious.
Agent-authored code is running into that lesson at 13 million sampled sessions from 3.2 million users in a single month on one vendor's coding agent alone, and the review layer is being assembled by app installation rather than design. We have been here before with humans signing off on AI-written code after outages, and with teams who trusted the scanner and shipped the vulnerabilities anyway. The failure mode is never that the tool was bad. It is that the reviewer deferred to something that sounded confident.
208,145 pull requests were graded by their author's own vendor. That is a supplier relationship, not a review.
Continue Reading
Zalando Auto-Approves a Third of PRs. Agents Made Them Bigger.
Agent Teams Hit 65 PRs a Week. Nobody Got Time Back.
Amazon's AI Code Created Outages. Now Humans Must Sign Off.
One Firm Ran Three Labs' Cyber Evals. Name Yours.
Anthropic's Evals Maxed Out. Stop Inheriting Its Assurance.
