Author: Prithvi Rajasekaran · Source: https://www.anthropic.com/engineering/harness-design-long-running-apps
Published: Mar 24, 2026 · Fetched: 2026-09-05
Note: body below is original English text extracted from the live page. Do not treat this file as a translation.
Harness design is key to performance at the frontier of agentic coding. Here's how we pushed Claude further in frontend design and long-running autonomous software engineering.
Over the past several months I've been working on two interconnected problems: getting Claude to produce high-quality frontend designs, and getting it to build complete applications without human intervention. This work originated with earlier efforts on our frontend design skill and long-running coding agent harness, where my colleagues and I were able to improve Claude's performance well above baseline through prompt engineering and harness design—but both eventually hit ceilings.
To break through, I sought out novel AI engineering approaches that held across two quite different domains, one defined by subjective taste, the other by verifiable correctness and usability. Taking inspiration from Generative Adversarial Networks (GANs), I designed a multi-agent structure with a generator and evaluator agent. Building an evaluator that graded outputs reliably—and with taste—meant first developing a set of criteria that could turn subjective judgments like "is this design good?" into concrete, gradable terms.
I then applied these techniques to long-running autonomous coding, carrying over two lessons from our earlier harness work: decomposing the build into tractable chunks, and using structured artifacts to hand off context between sessions. The final result was a three-agent architecture—planner, generator, and evaluator—that produced rich full-stack applications over multi-hour autonomous coding sessions.
Why naive implementations fall short
We've previously shown that harness design has a substantial impact on the effectiveness of long running agentic coding. In an earlier experiment, we used an initializer agent to decompose a product spec into a task list, and a coding agent that implemented the tasks one feature at a time before handing off artifacts to carry context across sessions.
But some problems remained persistent. For more complex tasks, the agent still tends to go off the rails over time. While decomposing this issue, we observed two common failure modes with agents executing these sorts of tasks.
First is that models tend to lose coherence on lengthy tasks as the context window fills. Some models also exhibit "context anxiety," in which they begin wrapping up work prematurely as they approach what they believe is their context limit. Context resets—clearing the context window entirely and starting a fresh agent, combined with a structured handoff that carries the previous agent's state and the next steps—addresses both these issues.
A second issue, which we haven't previously addressed, is self-evaluation. When asked to evaluate work they've produced, agents tend to respond by confidently praising the work—even when, to a human observer, the quality is obviously mediocre. This problem is particularly pronounced for subjective tasks like design, where there is no binary check equivalent to a verifiable software test.
However, even on tasks that do have verifiable outcomes, agents still sometimes exhibit poor judgment that impedes their performance while completing the task. Separating the agent doing the work from the agent judging it proves to be a strong lever to address this issue. The separation doesn't immediately eliminate that leniency on its own; the evaluator is still an LLM that is inclined to be generous towards LLM-generated outputs. But tuning a standalone evaluator to be skeptical turns out to be far more tractable than making a generator critical of its own work, and once that external feedback exists, the generator has something concrete to iterate against.
Frontend design: making subjective quality gradable
I started by experimenting on frontend design, where the self-evaluation issue was most visible. Two insights shaped the harness: First, aesthetics can be improved with grading criteria that encode design principles. Second, by separating frontend generation from frontend grading, we can create a feedback loop that drives the generator toward stronger outputs.
Four grading criteria given to both generator and evaluator: Design quality; Originality; Craft; Functionality. I emphasized design quality and originality over craft and functionality. I calibrated the evaluator using few-shot examples with detailed score breakdowns.
A generator agent first created an HTML/CSS/JS frontend. The evaluator used the Playwright MCP to interact with the live page before scoring each criterion and writing a detailed critique. That feedback flowed back to the generator. I ran 5 to 15 iterations per generation. Full runs stretched up to four hours.
Scaling to full-stack coding
The architecture
For this work I built on the foundation from the original harness with a three-agent system:
Planner: took a simple 1-4 sentence prompt and expanded it into a full product spec. Prompted to be ambitious about scope and stay focused on product context and high level technical design rather than detailed technical implementation.
Generator: instructed to work in sprints, picking up one feature at a time from the spec. Each sprint implemented the app with a React, Vite, FastAPI, and SQLite (later PostgreSQL) stack. The generator was instructed to self-evaluate at the end of each sprint before handing off to QA. It also had git for version control.
Evaluator: used the Playwright MCP to click through the running application the way a user would, testing UI features, API endpoints, and database states. It then graded each sprint against both the bugs it had found and a set of criteria covering product depth, functionality, visual design, and code quality. Each criterion had a hard threshold, and if any one fell below it, the sprint failed and the generator got detailed feedback.
Before each sprint, the generator and evaluator negotiated a sprint contract: agreeing on what "done" looked like for that chunk of work before any code was written. Communication was handled via files.
Running the harness
For a retro video game maker prompt:
| Harness | Duration | Cost |
|---|---|---|
| Solo | 20 min | $9 |
| Full harness | 6 hr | $200 |
The harness was over 20x more expensive, but the difference in output quality was immediately apparent. Solo run: game play broken. Full harness: planner expanded into a 16-feature spec across ten sprints; play mode worked.
Getting the evaluator to perform at this level took work. Out of the box, Claude is a poor QA agent. In early runs, it identified legitimate issues, then talked itself into deciding they weren't a big deal and approved the work anyway. The tuning loop was to read the evaluator's logs, find examples where its judgment diverged from mine, and update the QA prompt. It took several rounds before the evaluator was grading in a way that I found reasonable.
Iterating on the harness
Every component in a harness encodes an assumption about what the model can't do on its own, and those assumptions are worth stress testing as models improve.
With Opus 4.6, I removed the sprint construct. I kept both the planner and evaluator. Without the planner, the generator under-scoped. With sprints removed, the evaluator moved to a single pass at the end of the run rather than grading per sprint. On 4.5, the evaluator caught meaningful issues across the build. On 4.6, for tasks within the model's solo boundary the evaluator became unnecessary overhead; for tasks still at the edge, the evaluator continued to give real lift.
For a DAW prompt, V2 harness ran ~3 hr 50 min / $124.70. QA still caught real gaps (display-only features, stub recording, missing clip interactions) across multiple build/QA rounds.
What comes next
When a new model lands, re-examine a harness, stripping away pieces that are no longer load-bearing and adding new pieces for greater capability. The space of interesting harness combinations doesn't shrink as models improve — it moves.
Isolation note (for KB indexing)
In this design, the evolved object within a run is the application under construction (candidate). The planner/generator propose changes. The evaluator and promotion thresholds form the control plane. The harness orchestration itself is tuned offline by humans between experiments, not rewritten by the generator in the same run.