Rollback the World, Keep the Reflection: Rollback-Induced Reflection for Long-Horizon LLM Agents

Note: body below is original English text extracted from arXiv abs / HTML. Do not treat this file as a translation.

arXiv:2609.18304 · published 2026-09-16 · submitted 16 Sep 2026 (v1), last revised 17 Sep 2026 (this version, v2)

Abstract

Large language model (LLM) agents increasingly tackle long-horizon tasks through multi-step environment interaction, yet a single erroneous action can alter subsequent states and observations, causing errors to compound over time. Existing methods either correct the context without repairing altered environment states or restore earlier states while discarding useful experience, making it difficult to both eliminate failure conditions and avoid repeating past mistakes. We argue that reliable recovery should instead be treated as a rollback-boundary control problem that jointly determines when to intervene, where to resume, and what information should survive recovery. Based on this view, we propose Rollback-Induced Reflection (RIR), a unified recovery framework that restores execution to a selected prior state while carrying forward reusable knowledge distilled from the abandoned trajectory to guide subsequent decisions. We further characterize recovery through a unified operator over rollback depth and retained memory, providing a general view of state restoration and knowledge retention. Experiments on three long-horizon benchmarks demonstrate that RIR consistently improves task performance across multiple LLM backbones, with structured reflection memory preserving useful experience and selective rollback enabling efficient recovery.

Authors

Yi Yu , Liuyi Yao , Yaliang Li , Enshu Wang , Libing Wu

Key claims (verbatim-leaning English extract)

• Alibaba Group {yui1212,wanges17,wu}@whu.edu.cn {yly287738,yaliang.li}@alibaba-inc.com Enshu Wang Libing Wu Abstract

• Large language model (LLM) agents increasingly tackle long-horizon tasks through multi-step environment interaction, yet a single erroneous action can alter subsequent states and observations, causing errors to compound over time. Existing methods either correct the context without repairing altered environment states or restore earlier states while discarding useful experience, making it difficult to both eliminate failure conditions and avoid repeating past mistakes. We argue that reliable recovery should instead be treated as a rollback-boundary control problem that jointly determines when to intervene, where to resume, and what information should survive recovery. Based on this view, we propose Rollback-Induced Reflection (RIR), a unified recovery framework that restores execution to a selected prior state while carrying forward reusable knowledge distilled from the abandoned trajectory to guide subsequent decisions. We further characterize recovery through a unified operator over rollback depth and retained memory, providing a general view of state restoration and knowledge retention. Experiments on three long-horizon benchmarks demonstrate that RIR consistently improves task performance across multiple LLM backbones, with structured reflection memory preserving useful experience and selective rollback enabling efficient recovery.

• Large language model (LLM) agents can solve complex long-horizon tasks through reasoning, tool use, and environment interaction ( Wei et al., 2022 ; Yao et al., 2022 ; Park et al., 2023 ) . Yet such tasks are highly sensitive to erroneous actions: a single misstep can alter subsequent states and observations, causing later decisions to rely on corrupted context. Errors therefore compound over time and can drive the agent progressively away from a valid solution trajectory ( Hao et al., 2026 ) .

• Existing remedies intervene at two levels. Information-level methods append corrective feedback to guide future decisions ( Madaan et al., 2023 ; Shinn et al., 2023 ; Zhao et al., 2024 ; Kim et al., 2025 ) , but cannot undo environmental changes already caused by an erroneous action, and contaminated observations remaining in context may contradict the corrective

• advice itself. State-level methods instead restore execution to an earlier checkpoint and discard the erroneous suffix ( Zhou et al., 2023 ; Li et al., 2025 ; Zhang et al., 2026c ; Hao et al., 2026 ) , but state restoration alone does not specify what information from that suffix should survive. Retain too little and the agent may repeat the same failure; overgeneralize a local failure and viable alternatives may be incorrectly ruled out.

• A reliable rollback mechanism must therefore answer three coupled questions. First, when to roll back : is the current branch still productive exploration, or has

• recovery-control framework for long-horizon LLM agents. For when , Hybrid Adaptive Review combines agent-initiated and adaptive scheduled reviews

• to assess whether the current branch should continue or recover. For where , Coarse-to-Fine Restore Localization first narrows the search to a causally

• progress preservation. For what , Rollback-Consistent Reflection Memory separates branch-local state restored with the checkpoint from reusable knowledge that persists across rollback. The memory stores the stable task objective, reusable environment knowledge, past-attempt milestones, and conditioned failure analysis, while deliberately excluding the agent’s current state to avoid reintroducing stale claims after restoration.

• rollback depth k k and the memory ℳ + \mathcal{M}^{+} retained across the recovery boundary. We prove that common correction and rollback mechanisms are restricted cases of the unified operator, and that their induced recovery-policy classes are therefore contained within the RIR recovery space. This policy-class inclusion leads directly to an optimal-value monotonicity result, under which the best task-completion probability attainable by RIR is no lower than that of any such restricted mechanism. Empirically, RIR consistently outperforms representative baselines across three long-horizon benchmarks and two LLM backbones, improving average success rate by up to 6.57 percentage points while maintaining selective recovery under constrained interaction budgets. Our contributions are as follows:

• We propose Rollback-Induced Reflection (RIR) , which combines adaptive review, coarse-to-fine restore localization, and rollback-consistent reflection to recover execution state without discarding reusable knowledge from failed branches.

• Information-level correction. Self-Refine ( Madaan et al., 2023 ) , AgenTracer ( Zhang et al., 2026b ) ,

• across tasks. Although effective feedback can improve future behavior ( Huang et al., 2024 ; Kamoi et al., 2024 ) , information-level correction cannot

• typically focus on failure detection or restore-point selection rather than the full recovery boundary.

• In contrast, RIR treats the recovery boundary itself as an explicit control problem: it unifies state restoration and context reconstruction by jointly deciding when to recover, where to resume, and what information remains valid across the rollback boundary. Unlike methods that treat reflection or rollback in isolation, RIR restores execution while preserving reusable knowledge and excluding state claims invalidated by restoration.

• We consider a partially observable long-horizon interactive task ℰ = ( 𝒮 , 𝒜 , 𝒪 , P , Ω , R g ) \mathcal{E}=(\mathcal{S},\mathcal{A},\mathcal{O},P,\Omega,R_{g}) ,

• observation spaces. The environment evolves according to P ⁡ ( s t + 1 | s t , a t ) P(s_{t+1}|s_{t},a_{t}) and emits observations through Ω ⁡ ( o t + 1 | s t + 1 ) \Omega(o_{t+1}|s_{t+1}) . Given task objective g g , the outcome function R g ​ ( s T , y T ) R_{g}(s_{T},y_{T}) evaluates the terminal execution between the final state s T s_{T} and the ground truth state y T y_{T} , it is binary for verifiable tasks and

• At step t t , the base agent follows an LLM policy a t ∼ π θ ( ⋅ | g , h t , ℳ ) , a_{t}\sim\pi_{\theta}(\cdot|g,h_{t},\mathcal{M}), where ℳ \mathcal{M} is the persistent

• Reflection Memory and h t = ( o 0 , a 0 , … , a t − 1 , o t ) h_{t}=(o_{0},a_{0},\ldots,a_{t-1},o_{t}) is the interaction history of the current branch. The key distinction is that h t h_{t} is checkpointed with execution,

• recovery attempts. The action space consists of final responses, ordinary task tools, and a recovery-control

• tool: 𝒜 = 𝒜 resp ∪ 𝒜 tool , 𝒜 tool = 𝒜 task ∪ { rollback } . \mathcal{A}=\mathcal{A}{\mathrm{resp}}\cup\mathcal{A}{\mathrm{tool}},\mathcal{A}{\mathrm{tool}}=\mathcal{A}{\mathrm{task}}\cup{\texttt{rollback}}. Actions in 𝒜 task \mathcal{A}{\mathrm{task}} interact with the environment, while 𝒜 resp \mathcal{A}{\mathrm{resp}} terminates the episode with a final response.

• Rollback. Before each executable action, the system stores a checkpoint C i = ( s i , h i ) , C_{i}=(s_{i},h_{i}), containing the environment state and branch-local context at step i i .

• Let ℐ t = { i ≤ t ∣ C i ​ is restorable at step ​ t } \mathcal{I}{t}={,i\leq t\mid C{i}\text{ is restorable at step }t,} denote the set of admissible restore points. For a selected restore point r ∈ ℐ t r\in\mathcal{I}_{t} ,

• suffix : τ r : t = ( a r , o r + 1 , … , a t − 1 , o t ) . \tau_{r:t}=(a_{r},o_{r+1},\ldots,a_{t-1},o_{t}).

• a test-time recovery control policy Π \Pi . Given an agent-call budget B agent B_{\mathrm{agent}} and a rollback budget B rb B_{\mathrm{rb}} , RIR seeks to maximize

Structure (section headings from HTML)

• Rollback the World, Keep the Reflection: Rollback-Induced Reflection for Long-Horizon LLM Agents
• 1 Introduction
• 2 Related Work
• 3 Problem Formulation
• 4 Rollback-Induced Reflection Framework

• 4.1 Overview
  

• 4.2 When: Hybrid Adaptive Review
  

• 4.3 Where: Coarse-to-Fine Restore Localization
  

• 4.4 What: Rollback-Consistent Reflection Memory
  

• 4.5 Theoretical Analysis
  

• 5 Experiments

• 5.1 Experimental Setup
  

• 5.2 Main Results
  

• 5.3 Ablation Studies
  

• 6 Conclusion
• References

Remainder

Full original English text: see html_url / source_url / pdf_url in frontmatter.