Writing study: Do We Need Adam? Surprisingly Strong and Sparse Reinforcement Learning with SGD in LLMs

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Sagnik Mukherjee, Lifan Yuan, Pavan Jayasinha, Dilek Hakkani-Tür, Hao Peng · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Empirical analysis, diagnosis & auditing (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

18 physical pages

Before references / appendix

≈ 8.41 page-equivalents

Main figures / tables

4 / 6

Attached appendix figures / tables

2 / 1

Attached appendix span

≈ 2.91 page-equivalents

Before references / appendix: ≈ 46.72%References / intervening material: ≈ 37.11%Attached appendix: ≈ 16.17%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Source-bound correction: Source-checked physical-page text restores wrapped top-level heading labels on pp. 17, 18. Heading starts and measured positions are unchanged; this is label completeness, not a new section. Newly detected unnumbered impact/singular-acknowledgment statements are retained from the size-aware source extraction; authored heading corrections are unchanged.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 1.32 pp.	7.33%
2. Background	2	≈ 0.5 pp.	2.78%
3. Do We Need Adam?	3	≈ 1.04 pp.	5.78%
4. Can SGD Match AdamW in RLVR?	4	≈ 2.14 pp.	11.89%
5. SGD Induces Sparse and Low-rank Updates	6	≈ 0.92 pp.	5.11%
6. Validation with PPO and Extended Training	7	≈ 0.57 pp.	3.17%
7. Why are SGD Updates Sparser?	7	≈ 0.41 pp.	2.28%
8. Related Work	8	≈ 0.55 pp.	3.06%
9. Limitations and Future Work	8	≈ 0.22 pp.	1.22%
10. Conclusion	9	≈ 0.14 pp.	0.78%
Impact Statement	9	≈ 0.18 pp.	1%
Acknowledgements	9	≈ 0.05 pp.	0.28%
References	9	≈ 6.67 pp.	37.06%
A. Appendix	16	≈ 0.52 pp.	2.89%
B. SGD requires a high learning rate	16	≈ 0.82 pp.	4.56%
C. Experimental Details for ablating momentum and Adaptive Learning Rate	17	≈ 0.18 pp.	1%
D. Experimental Details for the Adaptive Learning Rate Analysis	17	≈ 0.47 pp.	2.61%
E. Experimental Details for the Momentum Analysis	18	≈ 0.33 pp.	1.83%
F. Learning Rate Sweep for AdamW and SGD	18	≈ 0.58 pp.	3.22%

All detected figure and table locations

Main text

• Table 1 · p. 2
• Figure 1 · p. 3
• Figure 2 · p. 3
• Figure 3 · p. 4
• Table 2 · p. 5
• Table 3 · p. 5
• Table 4 · p. 6
• Figure 4 · p. 7
• Table 5 · p. 7
• Table 6 · p. 8

Attached appendix

• Figure 5 · p. 17
• Figure 6 · p. 17
• Table 7 · p. 18

Argument and organization

1. Problem and contribution

   Questions an inherited optimization default by contrasting RLVR's changing policy/data and sparse reward signal with supervised next-token prediction.

2. Argument sequence

   A component-level optimizer table precedes the hypothesis, performance tests, sparsity analysis and explicit limitations; the order makes the control variables legible.

3. Evidence and verification

   The final appendix sweep keeps both optimizers' collapse-to-zero operating points, preventing the comparison from depending only on one favored learning rate.

Problem, claim and contribution

Questions an inherited optimization default by contrasting RLVR's changing policy/data and sparse reward signal with supervised next-token prediction.

Introduction and research-question box, pp. 1-3 · Main paper

Argument and section order

A component-level optimizer table precedes the hypothesis, performance tests, sparsity analysis and explicit limitations; the order makes the control variables legible.

Table 1 and Sections 2-10, pp. 2-9 · Main paper

Figures, captions and visual explanation

Figure 1 uses four aligned train/validation panels for math and code; Figure 2 places the SFT/RL distribution comparison beside the boxed question it motivates.

Figures 1-2, p. 3; PNG inspected · Main paper

Tables, comparisons and evidence

The final appendix sweep keeps both optimizers' collapse-to-zero operating points, preventing the comparison from depending only on one favored learning rate.

Appendix F and Table 7, p. 18; PNG inspected · Main paper

Appendix and reproducibility

Separate diagnostic appendices define optimizer-state capture timing and momentum alignment, while the training appendix distinguishes nominal learning rates across optimizers.

Appendices A and D-F, pp. 16-18 · Main paper

Clarity and writing conventions

The introduction presents mechanism explanations as hypotheses and partial attributions, while limitations explicitly name optimizer families not tested.

Introduction, p. 2; Section 9, p. 8 · Main paper

What to reuse—and what not to infer

Transferable lesson: Organize a diagnostic paper around separable components and preserve failed tuning settings alongside peak performance.

Boundary: Sparse-update percentages depend on the stated training/model setting; do not turn a selected minimum into a universal SGD property.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-focused distributed manuscript scan, not exhaustive technical/proof validation. Sections 1-10, pp. 1-9: framing, optimizer decomposition, model/domain comparisons, sparsity explanation, limitations and conclusion.

Appendix / supplement

Writing-focused appendix scan; code, data and external assets not executed. Appendices A-F, pp. 16-18: training settings, learning-rate rationale, diagnostic definitions and tuning sweeps; references pp. 9-16.

Visually inspected PDF pages

3, 18

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published ICML 2026 proceedings PDF, PMLR volume 306; retrieved 2026-10-07. Public OpenReview decision notes were not independently reverified. · 18 pages