CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agents

Note: body below is original English text extracted from arXiv abs / HTML. Do not treat this file as a translation.

arXiv:2609.18779 · published 2026-09-16 · submitted 16 Sep 2026

Abstract

Current Mixture-of-Agents (MoA) paradigms generally treat query routing and agent fine-tuning as separate processes, limiting their ability to respond to evolving agent capabilities. This disconnect prevents routing strategies from adapting to evolving agent capabilities during post-training and prevents agents from achieving synergistic data-driven specialization. To resolve this, we introduce CERA-MoA (Co-Evolving Router with continually learning Agents for Mixture-of-Agents), an iterative reinforcement learning framework where the dynamic router and independent agent policies co-evolve. We design a predictive familiarity estimator that leverages mid-layer hidden states to evaluate semantic competence among agents, avoiding the overhead of full rollouts. Based on these familiarity scores, a cumulative-threshold adaptive routing mechanism dynamically activates a tailored minimal agent subset, achieving a trade-off between task performance and efficiency. By proactively allocating targeted training samples to agents based on their evolving competence, CERA-MoA promotes capability differentiation. Extensive experiments across various domains demonstrate that CERA-MoA outperforms state-of-the-art static-agent routing and fix-workflow fine-tuning baselines.

Authors

Jiaxuan Jiang , Liyuan He , Zhixuan Fang

Key claims (verbatim-leaning English extract)

• Current Mixture-of-Agents (MoA) paradigms generally treat query routing and agent fine-tuning as separate processes, limiting their ability to respond to evolving agent capabilities. This disconnect prevents routing strategies from adapting to evolving agent capabilities during post-training and prevents agents from achieving synergistic data-driven specialization. To resolve this, we introduce CERA-MoA (Co-Evolving Router with continually learning Agents for Mixture-of-Agents), an iterative reinforcement learning framework where the dynamic router and independent agent policies co-evolve. We design a predictive familiarity estimator that leverages mid-layer hidden states to evaluate semantic competence among agents, avoiding the overhead of full rollouts. Based on these familiarity scores, a cumulative-threshold adaptive routing mechanism dynamically activates a tailored minimal agent subset, achieving a trade-off between task performance and efficiency. By proactively allocating targeted training samples to agents based on their evolving competence, CERA-MoA promotes capability differentiation. Extensive experiments across various domains demonstrate that CERA-MoA outperforms state-of-the-art static-agent routing and fix-workflow fine-tuning baselines.

• Although large language models (LLMs) have demonstrated remarkable reasoning capabilities, standard post-training methods frequently encounter generalization bottlenecks when dealing with increasingly complex and multifaceted problem domains ( Hong et al. 2024 ; Ye et al. 2025 ) . To transcend the limitations of single monolithic models, the Mixture-of-Agents (MoA) paradigm has emerged as a promising solution. By combining multiple agents, this paradigm aims to solve diverse tasks more effectively by leveraging the complementary strengths of different agents ( Wang et al. 2025a ) . Such collaborative frameworks have the potential to substantially expand the performance boundaries of LLMs on complex tasks.

• Despite this potential, the current development of multi-agent systems primarily bifurcates into two directions. The first direction focuses on optimizing orchestration mechanisms among fixed-capability agents, which is typically achieved through debate frameworks to refine reasoning consensus ( Chan et al. 2024 ; Estornell and Liu 2024 ; Yi et al. 2025 ; Hu et al. 2026 ; Fan et al. 2026 ; Qiao et al. 2026 ) , or through dynamic agent selection for efficient query allocation ( Xia et al. 2024 ; Yue et al. 2025 ; Lee et al. 2026 ; Poon et al. 2026 ; Wang et al. 2026a ; Xue et al. 2026a ; Wang et al. 2026c ) . The second direction explores training and fine-tuning agents, but typically operates within predetermined multi-agent workflow architectures ( Park et al. 2025 ; Motwani et al. 2025 ; Zhang et al. 2025 ; Xue et al. 2026b ; Zhao et al. 2026 ; Wang et al. 2026d ) . Although these approaches effectively enhance collective performance, they inherently decouple the routing strategy from the continual learning dynamics of agents.

• This decoupling exposes a critical research gap in contemporary multi-agent paradigms. First, the capabilities of individual agents continually evolve during the post-training phase. However, existing routing mechanisms are not designed to adapt to such capability shifts. Second, current post-training pipelines typically rely on manually partitioned datasets, lacking a dynamic sample allocation mechanism. Without routing specific training queries to agents based on their competence, the system fails to automatically induce distinct, targeted expertise. Consequently, the disconnect between routing strategies and the continual learning of agents prevents the system from achieving synergistic capability specialization, leaving individual agents acting as generalists rather than domain experts.

• To bridge this critical disconnect, we formulate Co-Evolving Router with continually learning Agents for Mixture-of-Agents (CERA-MoA), an iterative closed-loop paradigm that integrates agent learning with dynamic routing optimization. Rather than treating routing and agent adaptation in isolation, CERA-MoA not only adapts to the progressively shifting capabilities of individual agents, but also dynamically allocates tailored training queries to explicitly induce skill specialization. At the core of our system is a predictive familiarity estimator, which directly extracts the intermediate hidden states of LLMs to quantify how well an input query aligns with each agent’s learned expertise. This design provides a dynamic evaluation of relative competence prior to text generation, reducing the heavy computational overhead of external evaluators or full rollout generations. Utilizing these familiarity scores, we further introduce a cumulative-threshold adaptive routing strategy. Rather than relying on a fixed top- k k agent allocation, this mechanism dynamically selects a varying number of agents based on the estimated competence coverage of the agent population. By activating the smallest score-ranked subset of agents whose cumulative familiarity exceeds the threshold, our strategy achieves a fine-grained trade-off between task performance and computational cost.

• We introduce a sample-level Mixture-of-Agents framework that enables continual reinforcement learning of LLM agents through adaptive query routing. This co-evolutionary system simultaneously adapts the routing mechanism to shifting agent capabilities and dynamically allocates training queries thereby encouraging distinct problem-solving specialization.

• We design a predictive familiarity estimator for efficient semantic-level competence evaluation, together with a cumulative-threshold adaptive routing mechanism to dynamically balance performance and computational cost based on the estimated competence.

• We conducted extensive experiments demonstrating that our framework outperforms static-agent routing and fix-workflow fine-tuning baselines. Empirical results highlight performance gains across diverse problem domains.

• Multi-agent systems (MAS) have emerged as a powerful paradigm to extend the reasoning boundaries of large language models (LLMs) by leveraging collective intelligence ( Zhao et al. 2024 ; Ye et al. 2025 ) . To facilitate effective collaboration, standard MAS architectures typically organize models through structured interaction protocols, such as multi-agent debate ( Liang et al. 2024 ; Estornell and Liu 2024 ) , majority voting ( Chen et al. 2024 ; Taubenfeld et al. 2025 ) , or mixtures of independent agents ( Wang et al. 2024 ; Xie et al. 2025 ; Li et al. 2026 ) . Unlike token-level Mixture-of-Experts (MoE) architectures that route internal hidden representations across specialized sub-networks ( Oldfield et al. 2024 ; Lv et al. 2025 ; Zhuang et al. 2025 ) , MAS operates at the sample and semantic level, requiring high-level coordination among autonomous models. To optimize the collective efficacy of these collaborative systems, current research methodologies generally bifurcate into two directions: orchestration optimization , which focuses on dynamic interaction protocols among fixed-capability agents, and agent fine-tuning , which actively trains individual agent policies within predefined workflows.

• To optimize collaboration structure, existing literature widely investigates dynamic orchestration and query routing. Approaches range from multi-arm bandits ( Xia et al. 2024 ; Poon et al. 2026 ) and knapsacks within budgets ( Wang et al. 2025b ; Xue et al. 2026a ) to agent diversity maximization ( Xie et al. 2025 ) , lightweight evaluation scorers ( Yue et al. 2025 ; Wang et al. 2026a ; Wang et al. 2026c ) , and confidence-guided stepwise routing ( Lee et al. 2026 ; Wang et al. 2026b ) . Recently, studies have also explored leveraging LLMs directly as self-orchestrators ( Dang et al. 2026 ; Ke et al. 2026 ) , topology graph generators ( Zhang et al. 2026 ; Li et al. 2026 ) , or meta-thinkers ( Zhu et al. 2026 ) , while methods like AgentDropout ( Wang et al. 2025c ) prune redundant communication nodes to reduce overhead. Despite effectively optimizing orchestration and reducing costs, these techniques generally assume that candidate models remain static during the orchestration training phase. Consequently, they lack an integrated mechanism to realign query allocation as individual models actively evolve and specialize. CERA-MoA bridges this gap by co-evolving a predictive familiarity estimator that captures dynamic relative competence alongside agent policy updates, ensuring dynamic adaptation to shifting agent expertise.

• Beyond static interactions, recent work actively fine-tunes agents to enhance individual and collaborative reasoning using reinforcement learning, preference optimization, and supervised learning. The methods explore test-time self-verification ( Lee et al. 2025 ) , reasoning chain refinement ( Puerto et al. 2025 ) , self-reflection cycles ( Zhao et al. 2025 ) , and reflection interaction optimization ( Yuan and Xie 2025 ) . Within collaborative setups, fine-tuning is driven by rule-based verifiers ( Park et al. 2025 ) , tree-structured sampling ( Motwani et al. 2025 ; Zhao et al. 2026 ) , LLM-as-a-judge interactions ( Xue et al. 2026b ) , and end-to-end multi-agent reinforcement learning ( Wang et al. 2026d ) . However, existing fine-tuning pipelines predominantly optimize agent policies within predefined workflow architectures. Furthermore, they mainly rely on manually partitioned or uniform training datasets, which keep data allocation separate from real-time learning dynamics. Without capability-aware query routing during training, existing systems lack an explicit mechanism to guide agents toward complementary specialization, leaving agents to act as homogeneous generalists. CERA-MoA addresses this gap by integrating an iterative routing mechanism directly into the training loop, proactively allocating semantic queries to agents based on their evolving competence to explicitly foster domain specialization.

• As illustrated in Figure 1 , we formulate CERA-MoA ( C o- E volving R outer with continually learning A gents for Mixture-of-Agents) as a collaborative framework ℳ = ( 𝒟 ψ , { 𝒜 θ i } i = 1 N , 𝒮 ) \mathcal{M}=\left(\mathcal{D}{\psi},\left{\mathcal{A}{\theta_{i}}\right}{i=1}^{N},\mathcal{S}\right) , integrating a parameterized router 𝒟 ψ \mathcal{D}{\psi} , a population of continually learning agent policies { 𝒜 θ i } i = 1 N \left{\mathcal{A}{\theta{i}}\right}_{i=1}^{N} , and a voting-based aggregator 𝒮 \mathcal{S} . Our primary design objective is to establish a closed-loop co-evolutionary process for the router and the agents: adaptive query allocation routes targeted training samples to specific agents to drive domain specialization, while the router synchronously updates its evaluation parameters to accurately track these continually shifting agent capabilities.

• For an incoming query, the router 𝒟 ψ \mathcal{D}_{\psi} computes a familiarity score that measures relative agent competence and allocates the sample to a tailored subset of agents. Selected agents independently generate completions and optimize their policies through reinforcement learning, driven by task-specific rewards. Concurrently, the router evaluates the relative advantages of agent performance to update its familiarity estimator. This iterative feedback loop naturally encourages specialization: agents receive problems aligned with their potential, evolving from homogeneous generalists into domain specialists, while the router synchronously tracks their real-time expertise.

• During inference, 𝒟 ψ \mathcal{D}_{\psi} routes the query to the minimal subset of competent agents based on learned familiarity scores. Each activated agent generates a single response. For tasks with deterministic solutions, the aggregator 𝒮 \mathcal{S} executes familiarity-weighted majority voting:

• y ^ = arg ⁡ max ⁡ ∑ i ​ s.t. ans ​ ( a i ) = y y ⁡ f i ​ ( q ) , \hat{y}=\arg\max_{y}\sum_{i\text{ s.t. }\text{ans}(a_{i})=y}f_{i}(q), (1)

Structure (section headings from HTML)

• CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agents
• 1 Introduction
• 2 Related Work
• 3 CERA-MoA

• 3.1 Overview
  

• 3.2 Predictive Familiarity Estimator
  

• 3.3 Cumulative-Threshold Adaptive Routing
  

• 3.4 Joint Optimization of Router and Agents
  

• 4 Experiments

• 4.1 Experimental Setup
  

• 4.2 Main Results (RQ1)
  

• 4.3 Adaptation to Heterogeneous Models (RQ2)
  

• 4.4 Ablation on Familiarity Score (RQ3)
  

• 4.5 Ablation on Adaptive Routing (RQ4)
  

• 4.6 Capability Specialization Analysis (RQ5)
  

• 5 Conclusion
• References
• Appendix A Algorithm Pseudocode & Architecture Details

• A.1 Co-Evolutionary Training Procedure
  

• A.2 Inference and Consensus Aggregation
  

• Appendix B Detailed Experimental Setup & Reproducibility

• B.1 Hardware and Software Infrastructure
  

• B.2 Hyper-Parameter Configuration
  

• B.3 Prompt Design and Zero-Shot Task Instructions
  

• Appendix C Task-Specific Reward Formulation
• Appendix D Dataset Specifications

• D.1 Source Datasets
  

• D.2 Construction of Training and In-Distribution Test Sets
  

• D.3 Out-of-Distribution Evaluation
  

• Appendix E Additional Experiments and Analysis

Remainder

Full original English text: see html_url / source_url / pdf_url in frontmatter.