sethkarten/continual-harness: reference implementation (repo README)

Repo: https://github.com/sethkarten/continual-harness · Paper: arXiv 2605.09998

Reference implementation for Continual Harness (Karten et al., 2026), a reset-free framework that automates agentic-harness refinement through online in-context learning, evaluated on Pokémon Red and Emerald. This repository also contains the Gemini Plays Pokémon (GPP) benchmark harness used to complete Pokémon Blue, Yellow Legacy (hard mode), and Crystal — the first AI system to finish multiple Pokémon RPGs.

Continual Harness starts from a minimal environment interface (frames, ASCII text map, button inputs) and lets an LLM Refiner rewrite the full harness state — system prompt p, sub-agents G, skills K, memory M — in place mid-episode via CRUD edits on a trajectory window. The same loop extends to joint training of an open-source model's weights via an online DAgger + process-reward-model pipeline.

Provenance note in the README: the repo was previously published as sethkarten/pokeagent-speedrun, the reference framework for The PokéAgent Challenge (arXiv 2603.15563). It was renamed and generalized; the hand-engineered PokéAgent harness is retained as the pokeagent scaffold (H_expert), and the repo now centers on Continual Harness. GitHub auto-redirects the old URL.

What ships

• The continualharness scaffold — H_min plus an automated Refiner performing reset-free CRUD edits to prompt, sub-agents, skills, and memory mid-episode (the evolve_harness tool, gated by --enable-prompt-optimization). "This is the artifact for the paper's main results."
• The PokeAgent benchmark harness — hand-engineered H_expert with sub-agents, A* pathfinding, type chart, damage calculator, and curated objectives; the paper's upper baseline.
• Minimal baselines — simple (H_min) and simplest ablations.
• External CLI harnesses — Claude Code, Gemini CLI, Codex, and Hermes via an MCP proxy (run_cli.py), each run in Docker for isolation.
• Emulator integrations, persistence, metrics, and a web UI for both Pokémon Emerald (mGBA + Porymap) and Pokémon Red (PyBoy).

The refinement loop, as implemented

Every --optimization-window-length steps, a Refiner reads the recent trajectory window, identifies failure signatures (navigation loops, tool-call failures, stalled objectives, missed exploration), and runs four passes:

1. Rewrite the system prompt p against the observed failures.
2. CRUD over sub-agents G (create entries for repeated multi-step patterns, edit to address failures, delete unused entries).
3. CRUD over skills K (codify successful sequences, repair executable code that raised exceptions).
4. CRUD over memory M (fill gaps, refresh stale entries, demote areas the agent has moved past).

"Refinement is exposed to the agent as a single tool, evolve_harness, and the Refiner role uses the same model as the agent." Implementation: agents/utils/harness_evolver.py; scaffold wiring: agents/tools/registry.py.

# Continual Harness on Pokémon Red from the start (Gemini 3 Pro)
python run.py \
  --game red \
  --scaffold continualharness \
  --enable-prompt-optimization \
  --optimization-window-length 50 \
  --backend gemini --model-name gemini-3-pro-preview \
  --port 8000 --agent-auto

# Bootstrap from a prior run's evolved harness
# (loads memory.json, skills.json, subagents.json, evolved prompt)
python run.py \
  --game emerald \
  --scaffold continualharness \
  --enable-prompt-optimization \
  --bootstrap-from run_data/<prior_run_id>/end_state/game_state/bootstrap \
  --backend gemini --model-name gemini-3-pro-preview \
  --port 8000 --agent-auto

Without --enable-prompt-optimization the scaffold "behaves like simple plus the evolve_harness tool stub; full reset-free refinement only runs when the flag is set."

Paper variants → flags: from scratch = no --bootstrap-from, refinement enabled; bootstrap frozen = --bootstrap-from <path> without --enable-prompt-optimization; bootstrap updating = --bootstrap-from <path> with --enable-prompt-optimization.

Caveat stated in the README: "Capability-dependent gains: in the paper, Continual Harness is strictly Pareto-dominant on Gemini 3 Pro, high-variance on Flash, and below the minimalist baseline on Flash-Lite. Don't expect refinement to bootstrap below that capability floor."

Architecture

A headless server: the game and emulator run in a server process; agents and UIs are clients over HTTP REST plus POST /mcp/* routes.

• run.py starts the FastAPI game server on --port and, in agent mode, the frame stream server on --port + 1; the in-repo Python agent calls the game server HTTP routes directly.
• run_cli.py starts the game server, the frame stream server, and an MCP proxy on --mcp-sse-port (default --port + 2) for containerized CLI agents — "External CLI agents instead connect through the restricted MCP surface exposed by run_cli.py."
• The web UI is served at http://localhost:{port}/stream.

PokeAgent local subagents available to the hand-engineered harness: subagent_reflect, subagent_verify, subagent_gym_puzzle, subagent_summarize, subagent_battler, subagent_plan_objectives.

Backends: OpenAI, OpenRouter, Google Gemini, Anthropic, Vertex (utils/agent_infrastructure/vlm_backends.py). Requirements: Python 3.10–3.11, legally obtained ROMs (not included), mGBA system library for Emerald, PyBoy for Red.

Persistence detail worth noting: "Backups restore disk state under .pokeagent_cache/ (objectives, long-term memory, checkpoint, trajectories file if present, etc.), not the agent's in-memory short-term conversation window." Aggregate metrics source of truth is .pokeagent_cache/{run_id}/cumulative_metrics.json.

Agent-facing tool surface (from the shipped system prompt)

The continual-harness directive exposes memory and skill CRUD plus execution tools. process_memory(action=read|add|update|delete, entries=[...], reasoning=...) — hierarchical paths such as "locations/town_name", custom or auto-generated IDs, and a LONG-TERM MEMORY OVERVIEW tree of entry IDs injected into the prompt ("Use memory extensively"). process_skill(...) mirrors it, and skills "can be behavioral descriptions (text guidance) or executable code (Python that runs via run_skill)"; executable skills can call tools['press_buttons'], tools['get_game_state'], tools['get_map_data'], tools['complete_direct_objective'], and tools['process_memory'] in the same sandbox as run_code, returning data by setting a result variable.

The prompt also prescribes an explicit skill-development procedure — inspect with run_code + print() to learn the data structure, prototype one small behavior, iterate on tracebacks, save via process_skill(action="add", ...), then use via run_skill(skill_id=...) — with the standing instruction "Do NOT try to write complex code in one shot," and a requirement that looping skills re-read game state each iteration, detect stuck states, and exit after a max iteration count.

Full subagent tool set: press_buttons, get_game_state, get_map_data, complete_direct_objective, process_memory, process_skill, run_skill, run_code, process_subagent, process_trajectory_history, replan_objectives. Tool results arrive on the next step as a RESULTS FROM PREVIOUS STEP block.

Citation

@article{karten2026continual,
  title={Continual Harness: Online Adaptation for Self-Improving Foundation Agents},
  author={Karten, Seth and Zhang, Joel and Upaa Jr, Tersoo and Feng, Ruirong and Li, Wenzhe and Shi, Chengshuai and Jin, Chi and Vodrahalli, Kiran},
  journal={arXiv preprint arXiv:2605.09998},
  year={2026}
}

The README also asks users of the benchmark / pokeagent scaffold to cite The PokéAgent Challenge: Competitive and Long-Context Learning at Scale (arXiv:2603.15563).