Your Agent Aced the Task. Will It Do It Again?

Note: body below is original English text extracted from the source. Do not treat this file as a translation.

Published 2026-09-15 · https://huggingface.co/blog/ibm-research/altk-evolve-consistency · code https://github.com/AgentToolkit/altk-evolve · technical report https://arxiv.org/abs/2609.08832

Accuracy hides an unreliability problem. A ReAct agent (GPT-4.1 on AppWorld test_normal) that succeeds 77.4% of the time on average succeeds on all 5 repeated runs for only 53.0% of tasks — a 24.4-point consistency gap. On hard tasks it reaches 30 points.

We built a diagnostic for exactly this. The Consistency Analyzer resamples an agent's own recorded trajectory to find flip-prone decision points — steps where the model was one token-sample away from doing something different. It needs one trace and no ground truth — it resamples each decision point in that trace with a single call requesting k completions (k=5 by default), rather than re-running the task end-to-end.

Turning that diagnosis into guidelines halves the gap — from 24.4pp to 12.0pp (same-task Pass^5 +16.0pp, similar-task +13.0pp), without costing anything in average accuracy.

The Metric Almost Nobody Reports

Standard agent evaluation reports Mean@k: run a benchmark k times, average the pass rate. Often k=3, sometimes just 1. It's the number on every leaderboard, and it's what "77% accurate" means in practice.

Mean@k answers "how good is this agent, on average?" It does not answer the question a real user cares about: will it still be good if I ask this exact question again? For that you need Pass^k: the fraction of tasks where the agent succeeds on all k runs.

Pass^k is not Pass@k. The familiar Pass@k is optimistic — it asks whether at least one of k attempts succeeded, the right question when you can verify and retry. Pass^k is its pessimistic mirror image: every attempt must succeed. Same letters, opposite question. Pass^k ≤ Mean@k ≤ Pass@k, always.

A ReAct agent backed by GPT-4.1 posts a Mean@5 of 77.4% — genuinely strong. But Pass^5 is only 53.0%. Nearly a quarter of the benchmark consists of tasks the agent can sometimes solve and sometimes can't, with nothing about the task changing between runs. We call this gap — Mean@k minus Pass^k — the consistency gap.

This isn't a capability problem you fix with a bigger model. It's an orthogonal axis: an agent can be capable and inconsistent at the same time.

Why Agents Flip: Sharp Decisions vs. Flat Ones

Every time an LLM agent decides something — which API to call, what argument to pass, whether to retry — that decision comes out of a probability distribution over next tokens. What matters is the shape of that distribution. A sharp one puts most of its mass on a single token: the runners-up are far behind, and the same choice comes out run after run. A flat one spreads comparable mass across several near-tied tokens, and which one wins is close to a coin flip.

The shape decides how much noise it takes to change the outcome. Sharp distributions are resilient. Flat distributions are vulnerable. And because a trajectory chains dozens of decisions, a small per-step chance of flipping compounds into a large chance that some run goes differently.

Diagnose, Then Fix

Detect — the Consistency Analyzer. Given one recorded trajectory, the analyzer replays each decision step through controlled resampling, measuring how much the model's output actually varies at that point. Concretely, that's one additional model call per decision step, done once offline — issued with the sampling parameter set to draw k completions at once (k=5 by default) — replayed against the already-recorded context, not new tool calls, not new environment interactions, and not a second end-to-end rollout of the task.
Generate — targeted guidelines. Every flagged step becomes a candidate consistency guideline in the standard ALTK-Evolve format, so it slots into the existing storage and retrieval pipeline.

Results: Reducing the Gap Without Losing Accuracy

We evaluated on AppWorld test_normal (168 tasks) with a ReAct agent on GPT-4.1, generating consistency guidelines from a single baseline trajectory per task and testing them on 5 fresh runs.

The consistency gap is cut roughly in half. Aggregate Pass^5 rises 53.0% → 69.0% while Mean@5 rises 77.4% → 81.0%, narrowing the gap between "looks capable" and "can be counted on" from 24.4pp to 12.0pp. Nearly a third of previously-inconsistent tasks become tasks the agent passes on every single run.

The middle and hard tiers gain most. Medium +22.9pp (+44% relative), Hard +14.3pp (+45% relative). Easy gains +12.2pp. Mean@5 never drops.

Applied to a different but related task in the same AppWorld scenario, consistency guidelines still lift Pass^5 by +13.0pp, only 3 points below the same-task number.

If You're Shipping an Agent

Report Pass^k next to Mean@k.
Expect the gap to widen with difficulty.
Don't reach for a bigger model first. Consistency is orthogonal to capability.
Diagnosis needs no grader and no live replay. One extra LLM call per decision step (sampling k=5 completions by default) is enough.

Linked artifacts

ALTK-Evolve open source repo — github.com/AgentToolkit/altk-evolve
Technical report — arXiv:2609.08832

Remainder

Full original English text: see source_url / code_url / tech_report_url in frontmatter.