engineering leadership

SWE-Gate scored review-constraint compliance separately from functional tests across 303 repository-level repair tasks. Of 644 agent patches that passed the tests, 221 violated constraints taken from the original pull request reviews.

A study of 14,922 agent trajectories across five coding benchmarks finds Claude succeeds by matching the human patch's file scope while Qwen succeeds by exceeding it at every scale. Equal pass rates buy unequal diffs, and your ticket template is the control.

An ICSME 2026 replication across three matched model version pairs shows prompt technique effectiveness ages per model family, not uniformly. The playbook: add a stripped zero-shot control arm, re-evaluate every technique on each version bump, and put that checklist on the upgrade ticket.

A controlled crossover experiment with 34 inspectors found ChatGPT support was associated with about 8% lower specification defect-detection F1 while saving no time at all. The cohort that learned the task unaided first improved roughly 12% between periods against 7% for the cohort that started with the tool.

Canonical is funding three years of academic research into automated C-to-Rust translation. The published benchmarks explain why: the best result on a 100-repository benchmark is 48, and only with a repair loop, while researchers found just 69.9 percent of compiling translations behaved like the C they replaced.

A study of 248,641 agent-authored pull requests found 208,145 were reviewed by the same product that wrote them. If an agent's pass is your merge gate, you are counting one vendor twice.

Linear's first cross-company dataset tracked 6,887 teams for two years: those with a coding agent went from 21 pull requests a week to 65, those without went 8 to 10. Engineering time on triage, comments and review went up, not down.

Zalando published 2.5 years of agentic engineering data across 250+ teams. The 20-40% pull request lead-time win came from a bot that auto-approves 33% of PRs without a human — while PR sizes climbed into the 1k-2k line buckets and per-commit cyclomatic complexity showed inflection points exactly where coding agents entered.

On August 14 Claude Code defaults to auto mode on Pro, Max and Team plans. Anthropic's own docs say only permissions.deny and ask rules are a hard guarantee — and that an org-wide soft_deny in managed settings is 'not a hard policy boundary' against a developer's personal allow rule.

A vendor evaluation for VPs of Engineering buying 500+ seats: what each AI coding assistant actually costs once usage is metered, whose indemnity has a cap, which admin controls have documented holes, and why Amazon Q Developer is off the shortlist.