Alibaba's Qwen3.8-Max shipped with a long-horizon coding claim you can actually check — and checking it is the whole point. Every vendor selling you an autonomous coding agent this quarter will quote a number of hours or days it ran unattended. Almost none of them will hand you the artifact. Alibaba did: the repository the model allegedly built over 16 days is public on GitHub, Apache-2.0, with every commit, pull request and issue timestamped.
So I opened it. The timeline holds up. The headline number is softer than the coverage suggests, a named human wrote 2% of the commits and still owns the files that decide what merges, and the loop is doing something more mundane — and more interesting — than "16 days of engineering." None of that makes the claim false. All of it changes how you should run your next bake-off.
What Alibaba Claimed, and What GitHub Shows Today
Alibaba's claim is that Qwen3.8-Max ran an autonomous development loop for roughly 16 days and, by 30 July 2026, produced 265 commits, 127 pull requests and 151 issues on a CLI project called oh-my-cli, per developer-tech's write-up of the release. The model is a 2.4-trillion-parameter mixture-of-experts with 95 billion active parameters, according to The Decoder, carries a 1-million-token context window, and went generally available on 3 August 2026 at $2 per million input tokens and $6 per million output.
The first checkable fact is the calendar. GitHub's API reports the repository was created at 11:43:52 UTC on 13 July 2026, and the first pull request opened ninety-six minutes later. Alibaba's snapshot was taken on 30 July. That is 17 elapsed days of repository life containing a claimed 16-day run — consistent, not contradictory. The number that gets thrown around most is the one that survives contact with the timestamps.
The second checkable fact is that the loop did not stop when the press release went out. As of 5 August the repository holds 648 commits on main, 309 pull requests and 377 issues, against 620 stars and 54 forks. That is more than double the announced figures, six days past the snapshot they were measured at. The last push was at 21:11:57 UTC today. Whatever else is true, this is not a demo that was frozen for a screenshot.
Alibaba's Own Announcement Says "More Than Ten Days"
The 16-day figure and Alibaba's own wording are not the same claim. Alibaba's announcement describes "a single run lasting more than ten days" in which the model "built a software project called 'oh-my-cli' from scratch," and separately says that after roughly 16 days it had produced the 265/127/151 totals, as quoted by Trending Topics.
Steel-man it properly, because the honest reading is defensible: 10-plus days describes one uninterrupted build session, 16 days describes the elapsed window of the whole exercise. Those coexist. But they are different claims with different procurement implications, and the coverage collapsed them into one. Xinhua's report gave no duration at all, only "extended periods without human intervention." eWeek reported 16 days flat. TestingCatalog said "about 16 days."
If your vendor evaluation records "16 days unattended" in a spreadsheet cell, you have recorded the loosest available version of a claim whose author wrote something more careful. That is how a benchmark becomes a rumour in three hops.
A Human Wrote 13 of the 648 Commits
The repository has exactly two contributors. The bot account qwen-code-dev-bot has 635 commits. A human, qqqys, has 13. On pull requests the split is 302 to seven. So the run is overwhelmingly machine-authored — around 98% by either measure — which is a real result, and it is also not the same thing as zero human involvement.
Where the human intervened matters more than how often. Pull request #3, merged eighty seconds after it opened on day one, is titled "feat: add portable autonomous governance." A person built the control plane before the agent was allowed to run inside it. And pull request #501 — "fix(governance): support runner comment pagination" — landed at 08:53 UTC on 3 August, the morning of the announcement, because the hosted CI runner rejected the way the workflow was combining GitHub CLI pagination with query flags. A human was repairing the autonomy harness on the day the unattended run was being announced.
That is not a gotcha. It is the operating model, and the repository states it plainly. The CODEOWNERS file is four lines, and all four assign @qqqys: AUTONOMY.md, .autonomy/**, .github/workflows/**, and CODEOWNERS itself. The autonomy contract says the bot may read those paths and propose changes through governance issues but may never commit to them, may not change its own quality gates, must hold exactly one active lease and one mutation branch at a time, and may only execute issues authored by its own verified account. A merge requires local checks, GitHub checks, a clean current branch, complete ledger evidence, secret and dependency scans, and an independent self-review.
A long-horizon agent run is a control-plane design, not a duration. The interesting artifact here is not the 16 days. It is that somebody wrote down, in advance and in a file the agent cannot edit, exactly which decisions the agent was never allowed to make. Compare that to the production incidents where an agent had write access it was never scoped for, and the file is the more transferable output of the two.
What the Agent Was Building at Hour 500
Open the last few hours of the trace and the character of the work becomes obvious. Between 18:50 and 21:12 UTC today the bot opened and merged five pull requests: machine-readable exit-code signalling for the store doctor, then --strict exit-code signalling for the health report, then for the stale-sessions retention report, then for the attention summary, then a --follow live-watch mode for the workspace journal.
Four of those five are the same change applied to four different subcommands. Issue #684 was filed at 20:33:24, its pull request opened at 21:06:01, and it merged at 21:11:56 — 38 minutes from requirement to main, with under six minutes between opening the PR and merging it. Every one of the five merged in roughly five to six minutes.
That cadence tells you what a long-horizon agent actually is in production: not one enormous act of engineering sustained for 384 hours, but a very fast, very small loop that never gets bored, running against a queue it partly writes itself. It is closer to a tireless junior working a well-specified backlog than to an architect. That is genuinely valuable — most enterprise backlogs are 60% self-similar increments nobody wants to do — but it is a different purchase than the headline implies, and it argues for a different pilot design than the parallel-agent pitches built on raw agent count.
It also explains the model's own benchmark spread. Alibaba reports 86.6 on TerminalBench 2.1 and 93.0 on PaperBench, but 67.7 on SWE-Bench Pro against Fable 5's 80.0, per a compilation of the release numbers. Strong on long agentic loops, weaker on repository-scale bug fixing. The trace and the scores agree with each other.
Every Rival Long-Horizon Number Is Unfalsifiable
Here is the part that should change your evaluation template. Every headline figure in this release — the benchmark scores, the 125-hour paper reproduction, the simulated trading result — comes from Alibaba's own runs on Alibaba's own harnesses. No third-party benchmark authority, no audited enterprise deployment. That is normal. It is also true of nearly every long-horizon claim you will be shown this quarter.
The best independent long-horizon measure with a public methodology is METR's time-horizon work, which fits a logistic curve to agent success against how long a human expert takes the same task. Over its full history it has the 50% time horizon doubling roughly every seven months — a rate METR's January 2026 revision reconfirms for the long run while putting progress since 2023 nearer four months; its live leaderboard is the closest thing the field has to a referee. Note the units. METR measures in hours of human-equivalent task length, not in wall-clock days of a loop running. A vendor quoting days is not reporting a harder version of METR's number. It is reporting a different quantity that happens to sound bigger.
So the practical asymmetry is this: Alibaba's number is soft in the ways I have described, and it is still the most verifiable long-horizon claim on the market, because it is the only one that came with the artifact. A competitor who tells you their agent ran for three weeks and shows you a slide has given you strictly less than Alibaba gave you here. Reward the vendor who hands over the trace, even when the trace complicates their story — especially then. Anything else is scoring the vendor's own eval and calling it evidence.
Change Your Coding-Agent Bake-Off This Quarter
If you are running an evaluation on Cursor, Devin, Qwen Code or anything else that advertises unattended operation, three changes cost you nothing and remove most of the ambiguity.
This Week:
- Add one line to the trial agreement: the vendor supplies a complete commit trace — repository, or an escrowed export — for any long-horizon claim they make in the sales cycle. If they will not, the number does not go in the evaluation matrix. That single clause is the whole finding of this article.
- Rewrite every "ran autonomously for N days" cell in your current comparison sheet into three: elapsed wall-clock, longest uninterrupted session, and number of human commits during the window. Alibaba's own numbers give you 16, 10-plus and 13. Most vendors cannot fill in columns two and three, and that is the signal.
This Month:
- Ask each vendor for their equivalent of CODEOWNERS: the written list of paths, actions and decisions the agent is structurally prevented from touching. If the answer is a prompt instruction rather than an enforced file, it is not a control plane. Your own pilot needs one before the agent starts, not after the first bad merge.
- Measure the loop, not the run. Pull median time from ticket to merged PR and the share of merged changes that are near-duplicates of a previous change. On this repository those numbers are about 38 minutes and, in the last two hours, four in five. Those two figures predict the value you will get far better than a day count does.
Before Renewal:
- Re-run the trace check on the incumbent. Repository state moves — this one gained 383 commits in the six days after the snapshot Alibaba published — and a claim verified at signature is not a claim verified at renewal.
The Bottom Line
The last time the industry sold a capability by the hour, it was managed services, and the buyers who did well were the ones who audited the timesheet rather than the brochure. Long-horizon agents are the same purchase in new clothes, except the timesheet is a commit log and it is cryptographically ordered. That is a better deal than enterprise buyers usually get. Most of them are about to ignore it, sign on the day count, and discover the control plane the hard way — which is roughly why most agent pilots still never reach production, and why the honest comparisons are always the ones where nobody wins cleanly.
Alibaba published a number that its own repository partly undercuts. That is not a scandal. It is the first time you have been able to tell.
Demand the commit history. The benchmark is the vendor's story. The trace is yours.
Continue Reading
- DeepSeek Swapped the Model. Your Eval Didn't Notice.
- 64% of Fortune 500 Use AI Coding Agents. 33% Measure ROI.
- LangGraph vs CrewAI vs AutoGen: One of Them Is Retired
- Eleven Voice Agents, One Bank Call, No Clean Winner
- Blitzy's $1.4B Bet: 1,000 Coding Agents at Once
- Your AI Coding Tool Is Watching You. Alibaba Just Proved It.
- Stanford AI Index 2026: AI Agents Hit 66% Success Rate
- Why 88% of AI Agent Pilots Never Reach Production