second brain
source
← 首页

external-source

Warp 自进化 Agent：Improver 改 Skill 走 Git PR (Xudong Han)

Warp 自进化 Agent：Improver 改 Skill 走 Git PR (Xudong Han)

Source metadata

Post body

Claude 发文推荐 Warp 这套「自进化 Agent」做法，我觉得很值得看。

Agent 每次干完活，人类正常给反馈。另一个 Improver Agent 会定期把这些反馈捞出来，看它哪里反复犯错，再对原来的 Skill 提一个小修改。

修改直接走 Git PR，人 Review、Merge 以后，下一次 Agent 就会带着这次经验继续工作。

Warp 已经把这套机制用到了 Code Review、写 Spec 和 GitHub Issue Triage。

我觉得比较有趣的一点是，他们没有把学习理解成不停往 Prompt 里塞规则。

反馈最好告诉 Agent「为什么错」，Skill 也要保持小，只沉淀真正能复用的原则。

这种自进化其实很朴素：

Agent 干活 → 人纠正 → Agent 总结 → 修改自己的 Skill → 下一次少犯一次。

很多 Agent 现在缺的可能就是这个循环。

每次 Session 都积累了大量反馈，但下一次又像第一次来上班一样。

文章

Follow-up from the same author

warp的官推也发帖介绍了自己的自进化loop👇

Quoted Warp (@warpdotdev) — Thu, 27 Aug 2026 16:57:05 GMT — https://x.com/warpdotdev/status/2093019984993595698

Introducing self-improvement loops.

The concept is simple: what if agents could improve Skills by reviewing past conversations?

Here's the three step loop:

Linked essay 1 (full text)

Source: https://www.warp.dev/blog/self-improvement-loop-for-skills
Title: How to build a self-improvement loop for your Skills
Author: Zach Lloyd
Date: June 16, 2026

There's been a lot of chatter about using "loops" lately to drive agents, and I think this has been accompanied by a bit of "what actually is a loop"?

I can't speak for everyone else using the term, but I wanted to show a practical approach using Skills and cloud agents for a particularly powerful kind of loop: a self-improvement loop.

This is the idea that an agent can improve the quality of its own Skills over time from external feedback. My example is a loop that involves a human feedback step, but if you have a clear goal that doesn't require a human, you can use the same method with an automated grader.

To make matters concrete, say this Skill does issue triage, separating incoming issues into a few buckets: ready-to-implement, duplicate, needs-info. This would also work for a code review Skill, a bug fixing Skill, an incident response Skill, and so on.

Here's what a first draft of the Skill might look like:

Full triage-issue Skill

What you need to do is set up the following loops:

I'll show you how to do this in practice using Warp and Oz, our cloud agent platform, but there are lots of ways you can accomplish it. We will use Github Issues as the issue tracker.

Here is a sample repo with the Skills and GitHub workflows to follow along: https://github.com/warpdotdev-demos/issue-triage-loop

Step 1: set up the inner agent loop

The inner agent loop uses a Github action that runs on every new issue created.

Full GitHub Action

The Github action invokes a cloud agent through Oz, Warp's cloud agent platform. This cloud agent syncs the repo, pulls in the issue contents from github, and tries to classify it. The code on how to set this up is in the repo linked below.

Now when a new issue comes in, a cloud agent runs the inner loop triaging skill, and applies a label indicating that a new feature request is ready to implement.

Step 2: set up the outer loop for self-improvement

Let's say though that a human reviewer doesn't agree with the agent assignment. As a person looking at the agent's assigned labels, I switch the issue from "ready to implement" to "needs info" and add a comment on the thread as to why it was miscategorized, e.g. because there is ambiguity on whether we should add a setting for the new feature.

Here's where the outer loop becomes interesting. The outer loop agent runs once a day and looks at all issues that have been triaged, and when it runs, it will find that I manually adjusted the label and gave a reason why.

Full improve-triage-issue Skill

Since the outer loop agent Skill is run through a coding agent, it will take the feedback I provided and make a diff to update the triage Skill.

Once that diff merges, it feeds back into Skill that drives the inner loop agent, and the next time the agent runs the Skill should work better.

Would love to know if this is useful for folks. We use self improvement loops to manage the Warp open-source repository, and we extracted the framework behind it for others to adopt. Early version here.

Linked essay 2 (full text)

Source: https://www.warp.dev/articles/self-improvement-loop-software-factory
Title: What Goes Into a Self-Improvement Loop for a Software Factory?
Author: Warp
Date: August 24, 2026

A self-improvement loop in a software factory pairs scorers that grade finished agent runs against a rubric with observer agents that look for patterns across those grades, then writes the findings back into the factory's own code-based definition — its model choices, context, and skills — so the next run performs better than the last.

Why a factory needs to close the loop, not just run

Standing up a software factory moves triage, implementation, review, and verification off a person's plate — that's the automation half. The improvement half is different: it requires a closed-loop system that can look at what its own agents did, decide whether that was good, and change itself in response.

This is why factory metrics sit alongside DORA metrics rather than replacing them. DORA — deployment frequency, lead time for changes, change failure rate, mean time to restore — measures externally visible delivery. Factory metrics measure the inner loop that produced it: how many runs a task took, what they cost, how much human touch they needed. You only get factory metrics if a Warp factory — or any comparable closed-loop system — is capturing that trace to begin with; a set of interactive agents running on individual laptops leaves nothing comparable to grade.

The three components of a self-improvement loop

A working loop needs three distinct pieces, and it's easy to build only the first and call it done.

Scorers grade a finished run — the full conversation trace of a triage, implementation, or review agent, plus any human interaction with it — against a rubric. The grade can come from a human, from deterministic code, or from another agent acting as judge. Warp Factories ships default scorers for correctness, cost efficiency, and verbosity, but a Warp factory can define its own: did triage route the ticket to the right team, did it decide correctly whether a spec was needed.

Observers decide where scorers run: on what percentage of runs, on what cadence, batched how. Score every run and coverage is complete but expensive; sample and the signal gets cheaper and noisier. A common default is a fixed cadence — every few hours — rather than scoring continuously.

Self-improvement agents are the piece teams most often skip. Given a batch of scored runs — say, 100 graded runs of a triage agent — a self-improvement agent looks across them for where low scorers went wrong or high scorers did something worth keeping, and distills that into a pattern rather than an average.

How a scored run becomes a factory change

  1. An agent run completes and its trace, plus any human interaction, is captured.
  2. A scorer grades the run against a rubric — human, code, or LLM-as-judge.
  3. Scored runs accumulate by agent and by dimension.
  4. A self-improvement agent reviews a batch and identifies the pattern behind the failures or the wins.
  5. Because a Warp factory is defined as code, the fix comes back as a diff — a changed model choice, a rewritten skill, added context — against the factory definition itself.
  6. A human reviews and merges the change like any other PR.
  7. The next batch of runs is scored against the new baseline.

Step 5 is what separates a self-improving loop from a self-reporting one. A scorer dashboard tells you something is wrong; a mergeable diff tells you what to do about it.

Where the loop lives across software factory providers

Provider type Example Who defines the rubric Can you inspect or edit the loop Where the fix lands
Factory infrastructure Warp Factories You — default and custom scorers Yes, the factory is your own version-controlled code A diff against your factory definition, reviewed like any PR
Vertically-integrated factory product Factory.ai The vendor, inside its hosted product No — quality logic lives in their service, not yours The vendor's roadmap, on their release schedule
Point-solution automation Greptile The vendor, scoped to its one stage Partially — you see the output, not the scoring internals The vendor's product updates
Interactive coding agent Cursor, Claude Code, Codex No factory-wide rubric — each session stands alone Not applicable — no loop persists across sessions Nothing to diff; every session starts fresh

The pattern: a self-improvement loop needs somewhere for its fixes to land that you control. A hosted product can absolutely get better over time — but that improvement is the vendor's, running on their schedule, not a loop pointed at your own priorities.

What most teams get wrong

The most common failure is building the scorer and stopping there. A rubric that grades every run but never feeds a self-improvement agent is a monitoring dashboard, not a loop — it reports that triage is misrouting 15% of tickets without ever proposing a fix.

The second is scoring everything from day one. Scorers cost money to run; most factories get more signal per dollar sampling a percentage of runs on a fixed cadence and expanding coverage only where a metric looks off.

The third is keeping the factory definition itself outside version control. If the skills, context, and model choices an agent uses aren't code, a self-improvement agent has nothing to open a diff against — the loop has no mechanism to close, regardless of how good the scoring is.

Signs a self-improvement loop is actually working

If none of those are true yet, the loop hasn't closed — you have a scorer, not a self-improvement system.