From Question Answering to Task Completion: A Survey on Agent System and Harness Design
by Guo et al. (City University of Hong Kong, University of Sydney, Peking University)
The first systematic treatment of the harness — the runtime around the model that decides whether your agent finishes the task.
Overview
Seventeen authors from City University of Hong Kong, the University of Sydney, Peking University and TokenRhythm Technologies examine LLM-based agents through what they call a model-harness lens. The survey opens by separating functional definitions of an agent from implementation views, then traces four engineering paradigms in the order the field actually moved through them: prompt engineering, workflows and context engineering, harness engineering, and agent-native training. Its central contribution is decomposing the execution harness into six runtime responsibilities — observation, context, control, action, state and verification — which turns vague architecture debates into six separable design decisions with their own literature. From there it synthesises evidence on how runtime design affects task completion rate, efficiency and reliability, maps task properties onto harness configurations so the choice is driven by workload rather than fashion, reviews benchmark and evaluation methodology for agentic systems, and closes on open problems in safety, generalisation, and the co-evolution of models and the systems around them. The argument that agent behaviour emerges from the interaction of model capability, runtime design, task structure and evaluation methodology is the useful corrective for teams who respond to a failing agent by swapping the model. A companion GitHub repository, Awesome-Agent-Engineering, indexes roughly 250 papers under the same taxonomy — prompting and planning, workflows and tool use, harness and memory and protocols, benchmarks and safety, domain agents, and agent-native RL — so the survey doubles as a navigable map of the literature.
At a Glance
- Topic
- Agentic
- Level
- Advanced
- Format
- Paper
- Cost
- Free
- Duration
- ~2-3 hour read; longer if you follow the companion paper index
- Provider
- Guo et al. (City University of Hong Kong, University of Sydney, Peking University)
- Hands-on
- No
- Certificate
- None
What You’ll Learn
- ✓Separate a model's capability from its harness when diagnosing why an agent fails a task
- ✓Decompose any agent runtime into observation, context, control, action, state and verification responsibilities
- ✓Place your own stack on the four-paradigm arc from prompt engineering to agent-native training
- ✓Match harness configuration to task properties instead of copying whatever architecture is currently popular
- ✓Read agentic benchmark results critically, knowing which differences come from the runtime rather than the model
- ✓Identify where verification belongs in a pipeline and what it costs in latency and tokens
- ✓Navigate roughly 250 indexed papers through the companion Awesome-Agent-Engineering taxonomy
Highlights
- •Names and structures the harness layer that practitioner blog posts describe only anecdotally
- •The six-responsibility decomposition is directly usable as an audit checklist against an existing agent codebase
- •Published June 2026, so coverage includes agent-native training and current harness-engineering practice rather than 2024-era ReAct framing
- •Backed by a maintained GitHub index of ~250 papers organised under the same taxonomy as the survey
- •Explicitly argues against the common failure response of swapping models when the runtime is the actual bottleneck
Who It’s For
Best For
- ✓Engineers designing or debugging a production agent runtime rather than prompting an existing one
- ✓Researchers looking for a current, structured entry point into agent systems literature
- ✓Technical leads evaluating agent frameworks and needing criteria beyond feature lists
Prerequisites
- •Hands-on experience building at least one LLM agent with tool calling
- •Comfort reading academic survey papers and following citation trails
- •Familiarity with agent evaluation benchmarks such as SWE-bench or similar
FAQ
What is From Question Answering to Task Completion: A Survey on Agent System and Harness Design?
This June 2026 survey argues that agent performance is a property of the model and its harness together, not of the model alone, and then builds the vocabulary to reason about that claim. It is written for engineers and researchers who already build agents and keep discovering that the same model succeeds or fails depending on how the runtime feeds it context, mediates tools, and verifies results. Reading it gives you a decomposition you can audit your own agent stack against, plus a mapping from task properties to harness configuration.
Is From Question Answering to Task Completion: A Survey on Agent System and Harness Design free?
From Question Answering to Task Completion: A Survey on Agent System and Harness Design is free to access.
What level is From Question Answering to Task Completion: A Survey on Agent System and Harness Design for?
From Question Answering to Task Completion: A Survey on Agent System and Harness Design is aimed at a advanced audience. Recommended background: Hands-on experience building at least one LLM agent with tool calling, Comfort reading academic survey papers and following citation trails, Familiarity with agent evaluation benchmarks such as SWE-bench or similar.
How long does From Question Answering to Task Completion: A Survey on Agent System and Harness Design take?
Expect roughly ~2-3 hour read; longer if you follow the companion paper index. Most learners work through it at their own pace.
What will I learn from From Question Answering to Task Completion: A Survey on Agent System and Harness Design?
You'll learn: Separate a model's capability from its harness when diagnosing why an agent fails a task; Decompose any agent runtime into observation, context, control, action, state and verification responsibilities; Place your own stack on the four-paradigm arc from prompt engineering to agent-native training; Match harness configuration to task properties instead of copying whatever architecture is currently popular; Read agentic benchmark results critically, knowing which differences come from the runtime rather than the model; Identify where verification belongs in a pipeline and what it costs in latency and tokens; Navigate roughly 250 indexed papers through the companion Awesome-Agent-Engineering taxonomy.
Topics
Sources
This page was written from 3 sources, 2 on domains other than arxiv.org.