Harness design for long-running application development
by Anthropic
How Anthropic Labs built a planner/generator/evaluator harness that codes autonomously for hours — with the cost and wall-clock numbers.
Overview
Published 2026-03-24 by Prithvi Rajasekaran of Anthropic Labs, this is the follow-up to the November 2025 post 'Effective harnesses for long-running agents'. It opens with why naive implementations fail: models lose coherence as the context window fills, exhibiting what the author calls 'context anxiety' — wrapping work up prematurely as the perceived limit approaches — and they evaluate their own output far too generously. The piece argues for full context resets with structured handoffs over in-place compaction, and for separating evaluation from generation rather than trying to make a generator self-critical. It then applies a GAN-inspired generator-evaluator loop to frontend design, the hardest case because quality is subjective: four weighted grading criteria (design quality, originality, craft, functionality), few-shot calibration so the evaluator stops rubber-stamping, and an evaluator that drives the live page through Playwright MCP before scoring, over 5-15 iterations and 4+ hours of wall clock. The pattern is then scaled to full-stack work as three agents — planner, generator, evaluator — communicating through files, on a React/Vite/FastAPI/SQLite-or-Postgres stack with git, coordinated by 'sprint contracts' that define done before implementation starts. A head-to-head is reported: solo Claude Opus 4.5 at 20 minutes and $9 versus the full harness at 6 hours and $200. The post then removes the sprint construct once Opus 4.6 sustains longer tasks natively, retests on a DAW build at 3.8 hours and $124.70, and closes by arguing harness complexity should be re-examined at every model release.
At a Glance
- Topic
- Agentic
- Level
- Advanced
- Format
- Guide
- Cost
- Free
- Duration
- ~20 min read (~4,300 words)
- Provider
- Anthropic
- Hands-on
- No
- Certificate
- None
What You’ll Learn
- ✓Why context resets with structured handoffs beat in-place compaction on multi-hour runs
- ✓How to separate generation from evaluation so an agent stops grading its own work
- ✓Write weighted four-criterion rubrics that make subjective frontend quality gradable
- ✓Calibrate an evaluator with few-shot examples so it probes edge cases instead of approving
- ✓Structure planner, generator and evaluator agents that coordinate through files on disk
- ✓Negotiate sprint contracts that fix the definition of done before any code is written
- ✓Wire Playwright MCP so the evaluator exercises the running app the way a user would
- ✓Recognise when a harness component has become overhead and strip it after a model upgrade
Highlights
- •Publishes real cost and wall-clock numbers — $9/20min solo versus $200/6hr harnessed, and a $124.70 rebuild
- •Written by the Anthropic Labs engineer who built and tuned the harness, not a summary of it
- •Shows the harness being simplified after Opus 4.6 shipped, which is a rare argument against over-engineering
- •Names concrete failure modes — context anxiety, self-evaluation bias, rubber-stamping evaluators — and the fix for each
- •Independently covered by InfoQ, so the claims have had outside scrutiny rather than only vendor framing
Who It’s For
Best For
- ✓Engineers building agents intended to run autonomously for hours
- ✓Teams designing evaluation loops for output whose quality is subjective
- ✓Agent-platform architects choosing between compaction and context resets
- ✓Anyone deciding how much scaffolding a frontier model still needs
Prerequisites
- •You have already built an agent loop with tool use and hit its context limits
- •Familiarity with MCP and a browser-automation tool such as Playwright
- •Comfort reading multi-agent architecture descriptions without hand-holding
FAQ
What is Harness design for long-running application development?
An Anthropic engineering post describing the harness that lets a coding agent work autonomously for multiple hours without losing coherence or grading its own output too generously. It is written for engineers who already run an agent loop and need architecture, not an introduction. You come away able to design a generator-evaluator split, write rubrics that make subjective quality gradable, and decide which harness components to delete after a model upgrade.
Is Harness design for long-running application development free?
Harness design for long-running application development is free to access.
What level is Harness design for long-running application development for?
Harness design for long-running application development is aimed at a advanced audience. Recommended background: You have already built an agent loop with tool use and hit its context limits, Familiarity with MCP and a browser-automation tool such as Playwright, Comfort reading multi-agent architecture descriptions without hand-holding.
How long does Harness design for long-running application development take?
Expect roughly ~20 min read (~4,300 words). Most learners work through it at their own pace.
What will I learn from Harness design for long-running application development?
You'll learn: Why context resets with structured handoffs beat in-place compaction on multi-hour runs; How to separate generation from evaluation so an agent stops grading its own work; Write weighted four-criterion rubrics that make subjective frontend quality gradable; Calibrate an evaluator with few-shot examples so it probes edge cases instead of approving; Structure planner, generator and evaluator agents that coordinate through files on disk; Negotiate sprint contracts that fix the definition of done before any code is written; Wire Playwright MCP so the evaluator exercises the running app the way a user would; Recognise when a harness component has become overhead and strip it after a model upgrade.
Topics
Sources
This page was written from 2 sources, 1 on domains other than anthropic.com.