second brain
source
← 首页

external-source

RAFT: A Stateful Retrieval-Augmented Framework for Troubleshooting Agents

RAFT: A Stateful Retrieval-Augmented Framework for Troubleshooting Agents

Note: body below is original English text extracted from arXiv abs / HTML. Do not treat this file as a translation.

arXiv:2609.20754 · published 2026-09-17 · submitted 17 Sep 2026

Abstract

Effective troubleshooting agents in enterprise customer support depend on retrieving actionable guidance from similar historical cases, yet existing retrieval-augmented generation (RAG) systems treat support cases as static documents and overlook their multi-stage, stateful nature. We introduce RAFT (Retrieval-Augmented Framework for Troubleshooting Agents), a stateful RAG framework that abstracts each closed historical case into a directed chain of timeline entries and retrieves at the entry level, surfacing cases whose intermediate states match the active case and returning the parent-case trajectory anchored at the matched state; an optional case-level graph links cases through a configurable similarity representation. We evaluate this retrieval layer directly, which, unlike evaluating a full agent system, requires no production deployment. Because public multi-stage troubleshooting data is extremely rare, we pair a synthetic benchmark built from Microsoft Learn Windows Server documentation with real Apache Jira issues carrying human-created duplicate labels. RAFT improves Case Hit over vanilla RAG and GraphRAG baselines at every stage of case progress, with statistically significant gains over the strongest baseline; the Jira results provide directional evidence that the advantage transfers to real case histories. We release our benchmark, implementation, and the Apache Jira evaluation set.

Authors

Mingxuan Zhang, Xiaowen Wang, Anupma Sharan, Zhengyi Chen, Chenyu Diana Zhang, Shanshan Yang, Chittibabu Pacharu

Key claims (verbatim-leaning English extract)

Structure (section headings from HTML)

(section headings from HTML)

Body excerpts (original English)

Effective troubleshooting agents in enterprise customer support depend on retrieving actionable guidance from similar historical cases, yet existing retrieval-augmented generation (RAG) systems treat support cases as static documents and overlook their multi-stage, stateful nature. We introduce RAFT (Retrieval-Augmented Framework for Troubleshooting Agents), a stateful RAG framework that abstracts each closed historical case into a directed chain of timeline entries and retrieves at the entry level, surfacing cases whose intermediate states match the active case and returning the parent-case trajectory anchored at the matched state; an optional case-level graph links cases through a configurable similarity representation. We evaluate this retrieval layer directly, which, unlike evaluating a full agent system, requires no production deployment. Because public multi-stage troubleshooting data is extremely rare, we pair a synthetic benchmark built from Microsoft Learn Windows Server documentation with real Apache Jira issues carrying human-created duplicate labels. RAFT improves Case Hit over vanilla RAG and GraphRAG baselines at every stage of case progress, with statistically significant gains over the strongest baseline; the Jira results provide directional evidence that the advantage transfers to real case histories. We release our benchmark, implementation, and the Apache Jira evaluation set. 1 1 1 https://github.com/microsoft/RAFT Large language models (LLMs) now power intelligent agents across domains from software engineering ( Jimenez et al., 2023 ) to customer support ( Xu et al., 2024 ) . In enterprise customer support, an effective agent must reason over a private corpus of closed historical cases, whether it resolves incoming tickets or assists support engineers in doing so. Fine-tuning ( Hu et al., 2022 ; Ouyang et al., 2022 ) can inject such domain knowledge, but it is computationally expensive, restricted to open-weight models, prone to catastrophic forgetting ( Luo et al., 2023 ) , and requires periodic retraining as new and more recent cases emerge. Retrieval-augmented generation (RAG) ( Lewis et al., 2020 ; Gao et al., 2023 ; Singh et al., 2025 ) is a more practical alternative, grounding the LLM in a private knowledge base at inference time without altering its parameters. Yet as knowledge bases grow in scale and complexity, traditional RAG struggles: retrieved context is often extensive, poorly organized, and noisy, degrading both retrieval accuracy and the agent’s ability to reason over it ( Han et al., 2025b ; Edge et al., 2024 ; Xiang et al., 2025 ; Chen et al., 2024 ) . GraphRAG ( Edge et al., 2024 ; Zhang et al., 2025b ; Zhuang et al., 2025 ; Chen et al., 2025 ; Yang et al., 2026 ) responds by imposing explicit relational structure over the knowledge base. However, these general-purpose pipelines are not designed for troubleshooting histories, where investigations unfold across heterogeneous, noisy artifacts and closed cases vary in the actionable guidance they provide. Effective retrieval must identify relevant investigation states, preserve coherent case trajectories, distinguish useful evidence from non-actionable records, and protect sensitive information (Section 3 ). By contrast, entity-centric GraphRAG approaches build graphs of LLM-extracted entities and relations offline and use them to guide retrieval, without explicitly representing the progression of individual investigations. This adds graph-construction cost and couples retrieval to an entity-centric representation that can be harder to adapt as models and agent harnesses evolve ( Zhuang et al., 2025 ; Xiang et al., 2025 ; Chen et al., 2025 ) . We propose RAFT (Retrieval-Augmented Framework for Troubleshooting Agents), a stateful RAG framework built around the multi-stage nature of troubleshooting. RAFT abstracts each closed historical case into a directed chain of timeline entries, each recording the technical state at one meaningful stage of the investigation. Embedding and retrieving at the entry level rather than the case level surfaces cases whose intermediate states most closely match the active case, giving the agent both tactical guidance for the current stage and strategic context on where comparable cases led. A complementary, optional case-level graph links cases through a configurable similarity representation, enabling expansion to relevant cases that do not match at the entry level. To assess whether this representation supplies useful evidence throughout an investigation, we evaluate the retrieval layer independently of a complete troubleshooting agent. Realistic end-to-end evaluation often depends on access to operational environments and production workflows, making reproducible academic evaluation, cross-system comparison, and extension by others difficult. The retrieval layer, however, is independently testable across agent harnesses, underlying models, and workflows. We therefore measure whether, given the current information in an active case, RAFT surfaces similar historical cases that provide concrete evidence for diagnosis and resolution. This evaluation requires multi-stage troubleshooting histories and labels identifying similar cases, but suitable public data is extremely rare (Section 2 ). We evaluate on two complementary datasets: a synthetic benchmark constructed from Microsoft Learn Windows Server troubleshooting documentation ( MicrosoftDocs, 2024 ) for controlled demonstration and development, and real Apache Jira ( Apache Software Foundation, 2026 ) issues with human-created duplicate labels to check that the advantage transfers to real data. RAFT improves Case Hit over vanilla RAG and GraphRAG baselines at every stage of case progress (Sections 5.4 and 5.5 ). A stateful RAG architecture that represents closed historical cases as directed chains of timeline entries, retrieves over their evolving intermediate troubleshooting states, and returns the parent-case trajectory anchored at the matched state. A case-level graph linking cases through configurable case attributes, enabling principled expansion beyond initial entry-level matches. A public synthetic development benchmark of 826 support cases from Microsoft Learn Windows Server documentation, with a reproducible protocol that probes retrieval at multiple stages of an active case. For broad surveys of RAG, Agentic RAG, and GraphRAG, we refer readers to Fan et al. (2024) ; Zhang et al. (2025b) ; Singh et al. (2025) . These pipelines target generic document corpora, and applying them to historical customer support cases remains underexplored, largely due to the lack of suitable public datasets. To our knowledge, no public customer support dataset ( Abdellatif, 2025 ; Qu et al., 2018 ; Yang et al., 2018 ) satisfies all of the following criteria: (1) rich, multi-turn interactions rather than single-turn question answering; (2) key entities (error codes, products, services) preserved without redaction; and (3) labels grouping similar cases together. Beyond the data gap, most work in this area originates in industry settings where evaluation relies on proprietary data and production metrics, severely limiting reproducibility. Existing academic work also focuses largely on question answering over static domain documents ( Su et al., 2025 ; Patel, 2025 ; Zhao et al., 2025 ) rather than leveraging historical case interactions for multi-stage troubleshooting. The closest work to ours, Xu et al. (2024) , constructs a knowledge graph from historical cases for question answering and shows that graph structure can benefit support retrieval; however, their system is closed-source, their dataset private, and their evaluation confined to a production environment, making direct comparison infeasible. Evaluating the retrieval layer directly removes this dependence on production deployments and lets others reproduce and extend the comparison. We therefore pair a public synthetic benchmark for controlled demonstration and development with a transfer evaluation on Apache Jira ( Apache Software Foundation, 2026 ) , alongside an open implementation and a reproducible protocol. In enterprise customer support, particularly for IT services, resolving a ticket is rarely a one-shot process. Resolution unfolds across multiple stages: an initial symptom report, hypothesis formation and iterative information gathering across logs, system outputs, and diagnostic tools, and finally identifying the root cause and issuing targeted remediation. Across this trajectory, surfacing similar previously resolved cases and learning how they were diagnosed and fixed is critical for an agent to resolve the active case effectively. We therefore seek a retrieval system, built on a corpus of closed historical cases, that identifies similar cases given a query reflecting the current state of an active case and surfaces guidance on both the immediate next step and the broader resolution trajectory. Critically, retrieval must be state-aware : rather than treating the active case as a static lookup, the agent issues a sequence of updated queries as the active case evolves, and the system must return the most pertinent cases at each stage. Early on, a ticket describes only surface symptoms (an error code or a brief account of the behavior), and the agent should retrieve cases with similar initial symptoms to identify productive directions; as more context is gathered, updated queries let the system surface cases whose intermediate states match the current investigative state, giving finer-grained insight into the next steps. We deliberately scope this retrieval problem to surfacing similar cases and the evidence needed to diagnose and act on them. Higher-order reasoning, over how cases relate or how root causes cluster, is valuable but is best performed by the agent online against the specific active case rather than frozen into the offline index. We therefore keep retrieval focused on this task and defer relational inference to the agent, a stance our metrics (Section 5 ) reflect. This problem exposes four fundamental limitations of existing RAG approaches: Noisy, unstructured data. Raw case data interleaves substantial non-technical content with the diagnostically relevant details, and the signal is scattered across many turns. Because existing RAG baselines operate directly on this raw data, the noise degrades their ability to identify genuinely similar cases in the first place. Incoherent retrieved chunks. Even when a similar case is retrieved, an agent cannot reason effectively from isolated chunks. To judge whether a historical case is truly similar and to learn from how it was resolved, the agent must see the case as a coherent whole, which requires reassembling the raw case data by mapping retrieved chunks back to their parent cases. Given the size of real cases and the practical output-token limits of RAG tools, this both wastes tokens and sharply limits the number of similar cases an agent can examine per retrieval. GraphRAG methods that return chunks together with their entity relationships partially mitigate this, but the construction process is unstable and offers no guarantee that the chunks needed to form a coherent picture are retained ( Zhuang et al., 2025 ; Han et al., 2025a ; Zhou et al., 2025 ; Xiang et al., 2025 ) . Non-actionable cases. Closed tickets do not uniformly contain useful diagnostic or remediation evidence. Some end because the customer becomes unresponsive or the ticket is administratively closed, without documenting a meaningful investigation or outcome. A retrieval system must distinguish such records from cases that can inform the active investigation, rather than treating closure itself as evidence of usefulness. Privacy constraints. Enterprise customer data often contains personally identifiable information that should not be exposed directly to retrieval or to the agent, requiring an additional processing layer to abstract or redact such content before indexing. RAFT addresses these limitations through the two-level architecture described next. RAFT organizes closed historical cases at two levels: a per-case extracted resolution trajectory and a case-level graph 𝒢 \mathcal{G} that links cases through a configurable similarity representation. Let ℋ = { h i } i = 1 N \mathcal{H}={h_{i}}{i=1}^{N} be a corpus of N N closed historical cases. Each raw case h i h{i} has a unique identifier u i u_{i} (e.g., ticket number), metadata m i m_{i} (e.g., category, created and closed times), and a time-ordered turn sequence ( x 1 ( i ) , … , x T i ( i ) ) (x_{1}^{(i)},\ldots,x_{T_{i}}^{(i)}) of emails, notes, logs, and similar artifacts. We process each case independently to obtain a structured representation h ~ i \tilde{h}{i} : where ρ i \rho{i} is the reviewer assessment, { ϕ k ( i ) } k = 1 K i {\phi_{k}^{(i)}}{k=1}^{K{i}} the chronological timeline, r i r_{i} the root cause when established, a i a_{i} the documented resolution or mitigation steps, and e i e_{i} the troubleshooting-relevant entities. To construct this representation, workers process the ordered artifacts in bounded batches, carrying the evolving case state into each subsequent pass. Each worker receives the next batch alongside the metadata and accumulated state, adding new findings or revising earlier interpretations as evidence develops. Source-query and domain-specific tools provide additional evidence when needed. Once all batches have been processed, a reviewer checks and refines the completed state, consulting source evidence and revision history to resolve omissions or inconsistencies, and produces ρ i \rho_{i} . This workflow accommodates histories beyond a single context window while separating incremental extraction from final review. Further details of the extraction workflow are provided in Appendix B.2 . The application-specific assessment ρ i \rho_{i} can include case labels, actionability judgments, and supporting reasoning. Filters based on the extracted state (including entities e i e_{i} ), reviewer assessment ρ i \rho_{i} , and metadata m i m_{i} can be applied during indexing to omit cases from storage or at retrieval time to narrow the search space by error code, category, product version, timestamp, or case outcome. We denote the indices of cases retained in the search index by ℐ \mathcal{I} . Each timeline entry ϕ k ( i ) \phi^{(i)}{k} distills a contiguous segment of the case history, comprising consecutive turns ( x j ( i ) , x j + 1 ( i ) , … , x j + z ( i ) ) \big(x^{(i)}{j},x^{(i)}{j+1},\dots,x^{(i)}{j+z}\big) that together capture a meaningful stage of the investigation. These semantic segments need not coincide with the workflow’s processing-batch boundaries. Segmentation follows a single principle: a new entry begins at each meaningful state transition, where the framing of the problem or the current understanding is materially updated. These transitions correspond to the natural phases of an investigation, for example the opening symptom report, a hypothesis being added, discarded, or confirmed, the root cause being confirmed, and a resolution being proposed and verified. Acknowledgments and minor updates that add no new insight are absorbed into the current entry. This keeps the timeline compact ( K i ≪ T i K_{i}\ll T_{i} ) while ensuring every entry carries actionable information. Each entry records the technical state of its segment: the actions taken, the hypotheses under investigation, and the current understanding of the issue. Optionally, we construct an undirected graph 𝒢 = ( 𝒱 , ℰ ) \mathcal{G}=(\mathcal{V},\mathcal{E}) over stored cases, with vertex set 𝒱 = { h ~ i : i ∈ ℐ } \mathcal{V}={\tilde{h}{i}:i\in\mathcal{I}} . The text used to link cases is configurable: root cause, issue summary, or another deployment-specific field can be used alone or in combination. In our experiments, we concatenate root-cause and resolution texts and score every pair of cases using a hybrid of semantic similarity and BM25 lexical similarity ( Lù, 2024 ) , combined through Reciprocal Rank Fusion (RRF) ( Cormack et al., 2009 ) . We connect each case to its top- k k highest-scoring neighbors, symmetrize the result to obtain ℰ \mathcal{E} , and assign shared-nearest-neighbor (SNN) weights to the edges. Filters over metadata, entities, and reviewer assessments can further constrain the neighbor set during graph construction. The graph supports principled expansion beyond initial entry-level matches at retrieval time; outside this retrieval path, its communities can also support aggregate analysis of recurring issue families, although we do not evaluate that use here. We embed every timeline entry ϕ k ( i ) \phi{k}^{(i)} across all indexed cases ( i ∈ ℐ i\in\mathcal{I} ). Given a query q q , we first apply any user-specified case filter, then rank entries from the remaining cases using the same hybrid score (semantic plus lexical via RRF). We greedily promote ranked entries to their parent cases, selecting up to n n distinct cases within a predefined context budget. The full procedure is given in Algorithm 1 . For each selected case c ∈ C c\in C , k c ∗ k^{*}{c} is the index of the highest-scoring timeline entry, so the agent receives both the full case representation h ~ c \tilde{h}{c} and the anchor entry that triggered the match. This lets the agent see which investigation state was matched, not just which case. Once the agent has identified seed cases via entry-level retrieval, the case-level graph 𝒢 \mathcal{G} enables expansion to neighbors that are similar under the configured linking view but may not match at the entry level. For example, with the root-cause and resolution view used in our experiments (see Appendix C.5 ), linked cases may share an underlying cause or remediation strategy despite exhibiting different symptoms or intermediate states. Simply increasing the initial n n tends to introduce noise rather than uncover these complementary cases ( Zhang et al., 2025a ) . Graph expansion provides a targeted way to retrieve them while keeping the initial retrieval selective. We evaluate RAFT’s retrieval performance against vanilla RAG and GraphRAG baselines on a synthetic development benchmark, and then test whether the results transfer to real cases with human-created labels (Section 5.5 ). As discussed in Section 2 , no existing public dataset meets the requirements for evaluating retrieval in multi-stage troubleshooting. We therefore construct a synthetic corpus grounded in Microsoft Learn Windows Server troubleshooting documentation ( MicrosoftDocs, 2024 ) . The synthesis pipeline first organizes source articles into a structured wiki, then generates 2–4 support cases per documented root cause, injecting context from related articles to produce realistic diagnostic ambiguity. For evaluation, we hold out one case per root-cause group as the test query, with the remaining cases forming the indexed corpus. The synthetic data generation pipeline is detailed in Appendix A ; Table 1 reports per-category statistics. Category Cases Messages Tokens Active Directory 177 11.0 ± \pm 1.8 3297 ± \pm 795 Windows Security 121 9.7 ± \pm 1.8 2556 ± \pm 628 Remote Desktop 72 10.7 ± \pm 1.9 2794 ± \pm 657 Group Policy 92 9.8 ± \pm 2.2 2948 ± \pm 642 Licensing and Activation 62 11.1 ± \pm 1.9 2550 ± \pm 496 Networking 245 10.4 ± \pm 1.7 2346 ± \pm 579 Backup and Storage 57 10.4 ± \pm 2.2 3292 ± \pm 897 Overall 826 10.4 ± \pm 1.9 2767 ± \pm 770 We compare RAFT against vanilla RAG and two recent GraphRAG methods: HippoRAG2 ( Gutiérrez et al., 2025 ) and Fast-GraphRAG ( Circlemind AI, 2024 ) . HippoRAG2 builds an open-relation knowledge graph over the corpus and retrieves via personalized PageRank seeded by query-linked entities; Fast-GraphRAG extracts an entity graph and returns a budgeted mix of entities, relations, and chunks. We run both in their default configurations. To ensure a fair comparison, all methods share the same embedding model ( text-embedding-3-large ) and the same LLM for indexing ( gpt-5.2 ); evaluation uses gpt-5.4 . No metadata filtering is applied, isolating the effect of each method’s retrieval mechanism. Moreover, every case in our corpus is actionable, so RAFT’s actionability filtering excludes no cases and provides no advantage in this comparison. In production, retrieval tools exposed to an agent typically enforce a per-call token cap on returned context. Real support cases are token-intensive, often running into tens of thousands of tokens, whereas our synthetic cases are considerably shorter, averaging 2767 tokens (Table 1 ). To reflect this constraint, we cap retrieved context at 6000 tokens for all methods. For methods that return case-level units (RAFT, Vanilla RAG, HippoRAG2), we additionally cap retrieval at 5 distinct cases; Fast-GraphRAG returns entity-linked chunks rather than case-level units, so only the token cap applies. Detailed configurations are provided in Appendix C.2 . A key property of troubleshooting is that useful guidance depends on how far the investigation has progressed. To capture this, we construct queries from prefixes of each test case at three progress points: 0%, 30%, and 60% of turns. The 0% query contains only the initial symptom report; later cutoffs reveal progressively more diagnostic context. We report three metrics: Case Hit: whether at least one retrieved passage belongs to a ground-truth similar case, i.e., one sharing the same root cause and resolution. Root Cause Coverage: the fraction of atomic claims in the gold root-cause explanation that are entailed by the retrieved context, as judged by an LLM. Resolution Steps Coverage: the analogous fraction for the gold remediation procedure. Case Hit measures whether retrieval finds a matching case, while coverage measures how much of the gold diagnostic and remediation evidence the retrieved context supports. Full metric definitions appear in Appendix C.1 . Table 2 reports retrieval performance across all methods and progress levels. RAFT achieves the best scores on every metric, with the largest and most reliable gains on Case Hit. At 0% progress (initial symptom only), RAFT achieves 84.2% Case Hit compared to 67.3% for vanilla RAG and 65.0% for HippoRAG2. This advantage persists as more context becomes available, with RAFT reaching 88.8% Case Hit at 60% progress. To statistically evaluate this claim, we compare RAFT against vanilla RAG, the strongest baseline, using bootstrap resampling clustered by root-cause group; the Case Hit gains are statistically significant at all three progress points, with full confidence intervals in Appendix C.3 . Fast-GraphRAG underperforms vanilla RAG at all progress levels. This aligns with recent findings that complex entity extraction and graph-based reasoning offer little benefit, and can even degrade retrieval, when the task does not demand hierarchical knowledge retrieval or deep contextual reasoning across documents ( Xiang et al., 2025 ) . In our setting, surfacing similar cases and supplying actionable insights matter more than abstract relational inference. A similar pattern holds for HippoRAG2, which also fails to surpass vanilla RAG in this setting. To probe why entry-level retrieval helps, we examine where within a matched case the hit occurs. For each test case with a correct retrieval, we record the timeline entry that triggered the match and report its absolute index and its depth as a percentile of the number of timeline entries in the extracted case, averaged over hits, in Table 3 . The match moves steadily deeper as the query reflects a later stage, from 9.1% depth at 0% progress to 54.0% at 60%. This is the intended behavior: early queries carry only the symptom and match the opening entries of past cases, whereas later queries align with the corresponding intermediate states rather than re-matching symptoms. To assess robustness to noisy queries, we perturb test queries with off-topic content, typos, and dropout, and find that RAFT degrades less than vanilla RAG (Appendix C.4 ). In Appendix C.6 , we examine how indexing-model capacity affects RAFT’s retrieval performance through an ablation and present an agentic case study in which the agent further improves retrieval quality by composing its own queries and filtering conditions. Case Hit Root Cause Cov. Resolution Steps Cov. Method 0% 30% 60% 0% 30% 60% 0% 30% 60% Vanilla RAG 0.673 0.719 0.769 0.597 0.625 0.672 0.528 0.548 0.590 HippoRAG2 ( Gutiérrez et al., 2025 ) 0.650 0.688 0.711 0.574 0.609 0.637 0.507 0.541 0.559 Fast-GraphRAG ( Circlemind AI, 2024 ) 0.421 0.442 0.583 0.294 0.299 0.448 0.208 0.213 0.341 RAFT (Ours) 0.842 0.871 0.888 0.649 0.675 0.689 0.563 0.587 0.605 Case Matched-Entry Matched-Entry Progress Depth (%) Index 0% 9.1 0.28 30% 20.0 0.59 60% 54.0 1.58 To test whether RAFT transfers to real datasets, we build an evaluation set from public Apache Jira projects, where engineers link duplicate issues in their normal workflow (construction details in Appendix D ). The final evaluation set contains 30 audited duplicate groups, each evaluated against 570 additional resolved ( Fixed ) issues as distractors, over contributor-written, unredacted issue histories. Case Hit Method 0% 30% 60% Vanilla RAG 0.667 0.667 0.789 RAFT (Ours) 0.833 0.840 0.895 We apply RAFT as is, with no modifications to the extraction prompt, schema, models, or retrieval procedure from the synthetic experiments, and compare against vanilla RAG, the strongest baseline in Section 5.4 . RAFT improves Case Hit by + 16.7 +16.7 , + 17.3 +17.3 , and + 10.5 +10.5 percentage points at the 0%, 30%, and 60% progress points (Table 4 ), providing directional evidence that the advantage transfers to real case histories. We presented RAFT, a stateful retrieval-augmented framework that addresses the limitations of generic RAG and GraphRAG pipelines on enterprise customer support data. RAFT abstracts each closed historical case into a directed chain of timeline entries and optionally connects cases at the graph level through a configurable similarity representation, enabling state-aware retrieval that returns coherent, stage-specific evidence. On a synthetic benchmark built from Microsoft Learn Windows Server documentation, RAFT substantially improves Case Hit over vanilla RAG and recent GraphRAG baselines at every progress level, and an evaluation on real Apache Jira issues provides directional evidence that the advantage transfers to real case histories. While our experiments demonstrate the effectiveness of RAFT for retrieval, three limitations should be noted. First, our main evaluation is conducted on a synthetic dataset of moderate scale, whereas production corpora typically contain far more cases with substantially higher token counts per case. Second, the Apache Jira evaluation comprises 30 audited duplicate groups and carries no confidence intervals, so we treat it as directional transfer evidence rather than a comprehensive real-world evaluation. Third, this work does not evaluate final diagnosis, resolution success, engineer productivity, or other end-to-end troubleshooting outcomes; we scope the work to the retrieval layer for the reasons given in Section 1 . Microsoft Learn is Microsoft’s public documentation site for product guidance, troubleshooting articles, and learning resources. Each troubleshooting article typically documents a specific error or issue, including its symptoms, root cause, and resolution steps. Understanding these articles requires technical knowledge spanning multiple products, concepts, and technologies, which makes them a strong foundation for synthetic support case generation. We construct the dataset in two phases: we first build a knowledge base from the source articles, and then generate synthetic cases grounded in that knowledge base and troubleshooting articles. Rather than asking agents to read individual articles in isolation and generate cases from them, we first organize the source material into a wiki-style knowledge base. We focus on Windows Server troubleshooting and select seven categories: Active Directory, Group Policy, Licensing and Activation, Remote Desktop, Windows Security, Backup and Storage, and Networking. We use the Claude Code CLI with claude-opus-4-7 to construct this knowledge base, running one session per category. Each session spawns sub-agents that work on individual subcategories; each sub-agent reads all source files within its scope and classifies them as troubleshooting, informational, or general knowledge. The orchestrating agent then consolidates these results and produces a category overview.