Writing study: RAGEN-2: Reasoning Collapse in Agentic RL

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Zihan Wang, Chi Gui, Xing Jin, Qineng Wang, Licheng Liu, Kangrui Wang, Shiqi Chen, Linjie Li, Zhengyuan Yang, Pingyue Zhang, Yiping Lu, Jiajun Wu, Li Fei-Fei, Lijuan Wang, Yejin Choi, Manling Li · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Empirical analysis, diagnosis & auditing (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

33 physical pages

Before references / appendix

≈ 8.17 page-equivalents

Main figures / tables

9 / 3

Attached appendix figures / tables

5 / 7

Attached appendix span

≈ 19.57 page-equivalents

Before references / appendix: ≈ 24.76%References / intervening material: ≈ 15.94%Attached appendix: ≈ 59.3%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Source-bound correction: Source-checked physical-page text restores wrapped top-level heading labels on pp. 3. Heading starts and measured positions are unchanged; this is label completeness, not a new section. Newly detected unnumbered impact/singular-acknowledgment statements are retained from the size-aware source extraction; authored heading corrections are unchanged.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 0.9 pp.	2.73%
2. Template Collapse in Multi-turn Agent RL	2	≈ 1.57 pp.	4.76%
3. The Mechanism of Template Collapse: A Signal-to-Noise Ratio (SNR) View	3	≈ 1.97 pp.	5.97%
4. Experiments	5	≈ 1.58 pp.	4.79%
5. Analysis	7	≈ 0.96 pp.	2.91%
6. Related Work	8	≈ 0.48 pp.	1.45%
7. Conclusions and Limitations	8	≈ 0.27 pp.	0.82%
Acknowledgements	9	≈ 0.05 pp.	0.15%
Impact Statement	9	≈ 0.08 pp.	0.24%
References	9	≈ 5.26 pp.	15.94%
A. Extended Related Work	14	≈ 1.29 pp.	3.91%
B. Additional MI proxies	15	≈ 0.11 pp.	0.33%
C. Detailed Experimental Settings	15	≈ 1.73 pp.	5.24%
D. Filtering Ablation Results	17	≈ 1.04 pp.	3.15%
E. Additional Experimental Visualizations	18	≈ 0.19 pp.	0.58%
F. Notation and basic identities	18	≈ 1.7 pp.	5.15%
G. Scorer-based Proxies for Reasoning Diversity	20	≈ 0.59 pp.	1.79%
H. Formal Definition of the Filtering Operator	21	≈ 1.33 pp.	4.03%
I. RV Controls Task-Signal Magnitude and SNR	22	≈ 2.71 pp.	8.21%
J. Template Mixing Reduces Input Dependence	25	≈ 0.67 pp.	2.03%
K. Filtering Reduces Gradient-Estimation MSE	25	≈ 1.02 pp.	3.09%
L. Reward-Agnostic Regularizers and Update Dominance	26	≈ 0.7 pp.	2.12%
M. KL-Closeness to the Base Implies MI-Closeness	27	≈ 0.89 pp.	2.7%
N. Decomposing Changes in Input Dependence	28	≈ 0.46 pp.	1.39%
O. GRPO Normalization Amplifies Noise at Low RV	28	≈ 0.74 pp.	2.24%
P. Core Author Contributions	29	≈ 0.12 pp.	0.36%
Q. MI Proxy Family (extended table)	29	≈ 0.06 pp.	0.18%
R. Filtering Strategy Ablations	29	≈ 0.83 pp.	2.52%
S. Stress-Testing the SNR Mechanism	30	≈ 2.76 pp.	8.36%
T. Extended Limitations	33	≈ 0.14 pp.	0.42%
U. Core Author Contributions	33	≈ 0.5 pp.	1.52%

All detected figure and table locations

Main text

• Figure 1 · p. 3
• Figure 2 · p. 4
• Figure 3 · p. 4
• Table 1 · p. 5
• Table 2 · p. 5
• Figure 4 · p. 6
• Figure 5 · p. 6
• Figure 6 · p. 6
• Table 3 · p. 7
• Figure 7 · p. 7
• Figure 8 · p. 8
• Figure 9 · p. 8

Attached appendix

• Table 4 · p. 16
• Table 5 · p. 18
• Figure 10 · p. 19
• Table 6 · p. 29
• Table 7 · p. 30
• Figure 11 · p. 30
• Table 8 · p. 31
• Table 9 · p. 31
• Table 10 · p. 31
• Figure 12 · p. 32
• Figure 13 · p. 32
• Figure 14 · p. 33

Argument and organization

1. Problem and contribution

   Separates within-input diversity from dependence on the actual input, making high-entropy but input-agnostic reasoning a distinct failure mode.

2. Argument sequence

   Defines the phenomenon and measurable proxies before a gradient signal/noise account, then connects that account to reward-variance filtering and intervention sweeps.

3. Evidence and verification

   Prompt/trajectory filtering, noise injection and regularizer sweeps test the diagnosis, while the main conclusion states the assumed signal/noise separability and sparse/noisy-reward boundary.

Problem, claim and contribution

Separates within-input diversity from dependence on the actual input, making high-entropy but input-agnostic reasoning a distinct failure mode.

Section 2 and Figure 1, pp. 2-3 · Main paper

Argument and section order

Defines the phenomenon and measurable proxies before a gradient signal/noise account, then connects that account to reward-variance filtering and intervention sweeps.

Sections 2-5, pp. 2-8 · Main paper

Figures, captions and visual explanation

The quadrant diagram contrasts entropy and input dependence; the intervention figure holds axes constant across entropy, KL and filter sweeps and marks direction between two checkpoints.

Figure 1, p. 3; Figure 8, p. 8; both PNGs inspected · Main paper

Tables, comparisons and evidence

Prompt/trajectory filtering, noise injection and regularizer sweeps test the diagnosis, while the main conclusion states the assumed signal/noise separability and sparse/noisy-reward boundary.

Section 7, p. 8; Appendix S, pp. 30-32 · Main paper

Appendix and reproducibility

Definitions distinguish fixed-scorer log-likelihood proxies from true mutual information; extended limitations retain reward-variance gaming and task-specific filtering choices.

Appendices F-G, pp. 19-20; T, p. 33 · Main paper

Clarity and writing conventions

The metric names are memorable, but their probabilistic interpretation is qualified more precisely in the appendix than in the high-level narrative.

Section 2, pp. 2-3; Appendix G, p. 20 · Main paper

What to reuse—and what not to infer

Transferable lesson: Name a failure mode with two distinguishable axes, then connect the diagnostic to interventions and explicit stress tests.

Boundary: The MI quantities are scorer-dependent proxies, not direct estimates of true information; the filter keeps cumulative variance mass, not a fixed fraction of prompts.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-focused distributed manuscript scan, not exhaustive technical/proof validation. Sections 1-7, pp. 1-8: input-grounded versus templated reasoning, proxy definitions, gradient signal/noise diagnosis, filtering intervention, sweeps and limitations.

Appendix / supplement

Writing-focused appendix scan; code, data and external assets not executed. Appendices A-U, pp. 14-33: proxy variants, setup, gradient/regularizer derivations, filtering stress tests, additional results and extended limitations.

Visually inspected PDF pages

3, 8

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published ICML 2026 proceedings PDF, PMLR volume 306; retrieved 2026-10-07. Public OpenReview decision notes were not independently reverified. · 33 pages