Capability
Keeps required expertise covered, preventing evolution from pruning the last role for a task family.
Problem & formulation
Abstract
Role-based LLM multi-agent systems need adaptive role pools, yet adapting such systems is not merely a matter of prompt optimization: roles often carry structural obligations, including capability coverage, message compatibility, validation, final-answer aggregation, and parser-compatible output protocols. Existing systems either fix the role inventory and lose adaptivity, or allow unconstrained generation to induce role drift, removing structurally necessary roles and breaking answer contracts. We formulate this as contract-preserving role evolution, requiring every committed edit to preserve five structural contracts. We instantiate this formulation in Sero, a Self-Evolving Role Orchestration framework that evolves a typed role-card pool through credit-guided retrieval, a credit-ranked communication DAG with a protected terminal aggregator and conditional validator repair, and a contextual-bandit controller whose LLM-proposed edits are committed only when they preserve the contracts and improve task score. Experiments on three real-world reasoning benchmarks — NaturalPlan, TableBench, and OlympiadBench — across three LLM backbones confirm the value of contract-preserving role evolution.
Structural rails
Five Structural Contracts
Every committed edit to the role pool must preserve all five contracts, so the pool remains executable end-to-end.
Communication
Keeps intermediate states exchangeable under declared protocols, preventing incompatible or disconnected handoffs.
Validation
Keeps error-detection or repair capacity reachable, preventing unchecked drafts from bypassing critique.
Aggregation
Keeps a protected terminal role responsible for the final decision, preventing competing or missing answers.
Output Protocol
Keeps the final response compatible with the benchmark parser or application interface.
Credit-conditioned evolution
Method
Sero couples multi-scale credit with contract-guarded persistent editing: every candidate must preserve the five contracts and pass the score gate before it can change the role pool.
Framework overview of Sero. ① A shared inference operator retrieves an active team, runs a credit-ranked DAG ending at a locked aggregator with optional validator repair. ② A typed role pool stores cards with a locked validator and aggregator anchoring the five contract rails. ③ The controller proposes an Add/Remove/Noop edit, which is committed or rolled back by the score gate. Paper Figure 2
1
Typed Role Cards
Each editable unit stores its prompt, capability metadata, communication protocol, decoding temperature, role type, and protection flag.
2
Multi-Scale Credit
Fast within-run, periodic leave-one-out, and historical EMA credit jointly guide retrieval, retention, and removal.
3
Credit-Ranked DAG
A query-conditioned team runs through a stage-aware DAG that terminates at a protected aggregator, with optional validator repair.
4
Guarded Evolution
A contextual-bandit controller proposes Add, Remove, or Noop; contract checks and a phase-aware score gate decide what persists.
Three backbones · three seeds
Main Results
Sero is the top method on every metric for GPT-4o-mini and Gemini-2.5-flash-lite, and remains best on OlympiadBench and TableBench under Qwen3-8b.
+1.51
GPT-4o-mini
Average-score gain over Static Role Orchestration, the strongest baseline by average.
+5.51
Gemini-2.5-flash-lite
Average-score gain over Static DAG MAS, the strongest baseline by average.
2 / 4
Qwen3-8b
Best on OlympiadBench and TableBench; second by row average. This is the paper's stated scope boundary.
Swipe horizontally to compare all metrics.
Scores are averages over three seeds, with standard deviations in gray. Best and second-best per column. Δ is Sero's gain on the row average (positive green, negative red). 🥇 / 🥈 mark the top-1 and top-2 method per backbone by average score.
Component study · seed 44
Ablation Study
Removing credit yields the largest NaturalPlan drop because credit jointly conditions edit decisions, active-role ranking, and DAG ordering.
Swipe horizontally to compare all ablations.
Configurations: ✓ enabled · ◐ default/partial · ✗ disabled · 🎓 learned · ❄️ frozen pool · 🎲 random · ▶ forward-only. Δ is the absolute drop relative to the full method. This study uses seed 44.
Mechanism evidence
Analysis
The gains track backbone capacity, reusable coordination templates, and task-conditioned specialization—not a larger controller or more inference calls.
Scaling Behavior
Base-model scaling lifts all three benchmarks as Qwen3 grows from 1.7B to 32B (seed 43); controller-width scaling is comparatively flat (seed 44). Paper Figure 5
Reusable Active-Set Signatures
A few dominant signatures capture most routing decisions while retaining a stable Reasoning + Validation + Synthesis core. Paper Figure 4
Hyper-parameter Sensitivity
Sero stays competitive across one-at-a-time sweeps of specialist budget, credit-EMA momentum, and collaboration rounds; the optimum is benchmark dependent. Paper Figure 3 · seed 44
Compute Overhead
Sero matches Static DAG MAS at 6.31 calls per instance, adds 250.66 tokens on average, and improves the cross-backbone average by 2.83 points.
Swipe horizontally to compare systems.
Persistent Evolution & Task-Conditioned Routing
Appendix analyses show that committed roles can persist and re-enter later inference graphs, while specialist activation changes with the task group.
Role Lifecycle Across Training
A representative Gemini-2.5-flash-lite trajectory traces edit decisions, persistent role states, and active-role families over training. Appendix Figure 21 · seed 42
GPT-4o-mini Specialist Activation
Math and physics groups activate specialist roles at different rates; mean inter-group gap is 0.206. Appendix Figure 22 · seed 42
Qwen3-8b Specialist Activation
Task-conditioned specialization remains visible with a smaller active team; mean inter-group gap is 0.346. Appendix Figure 23 · seed 44
What the evidence covers
Current Scope
The paper is explicit about what has—and has not—been established.
Evaluated setting
- Text-only structured-reasoning benchmarks with automatically scorable outputs.
- Three frozen LLM backbones; only the lightweight role-editing controller is trained.
- Hand-designed seed role pools organized by capability family.
Open directions
- Multimodal and multilingual reasoning.
- Domains with sparse or subjective rewards.
- End-to-end training-cost accounting at larger backbone scales.
Reference
Citation
Accepted to the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP 2026). Preprint: arXiv:2605.28433.
@article{ge2026roles,
title={Roles with Rails: Contract-Preserving Role Evolution in Multi-Agent Structured Reasoning},
author={Ge, Ling-Yue and Guo, Lan-Zhe},
journal={arXiv preprint arXiv:2605.28433},
year={2026}
}