Writing study: MuonSSM: Orthogonalizing State Space Models for Sequence Modeling

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Thai Khanh Nguyen, Vo Ngoc Bich Uyen, Thieu Vo, Tan Minh Nguyen, Cuong Pham · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Architecture & representation (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

22 physical pages

Before references / appendix

≈ 9.18 page-equivalents

Main figures / tables

5 / 7

Attached appendix figures / tables

2 / 5

Attached appendix span

≈ 9.86 page-equivalents

Before references / appendix: ≈ 41.73%References / intervening material: ≈ 13.45%Attached appendix: ≈ 44.82%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Source-bound correction: The source uses Appendix A: rather than A.; restore its actual physical-page boundaries from the PDF text and glyph positions. Main references start p.10; supporting material starts p.13. Newly detected unnumbered impact/singular-acknowledgment statements are retained from the size-aware source extraction; authored heading corrections are unchanged.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 0.94 pp.	4.27%
2. Methodology: MuonSSM	2	≈ 1.11 pp.	5.05%
3. Theoretical Analysis	3	≈ 1.87 pp.	8.5%
4. Experiments	5	≈ 1.61 pp.	7.32%
5. Empirical Analysis	6	≈ 1.94 pp.	8.82%
6. Related Work	8	≈ 0.9 pp.	4.09%
7. Concluding Remarks	9	≈ 0.36 pp.	1.64%
Impact Statement	10	≈ 0.05 pp.	0.23%
Acknowledgements	10	≈ 0.09 pp.	0.41%
References	10	≈ 2.95 pp.	13.41%
Appendix A: Proofs of Theoretical Results	13	≈ 5.71 pp.	25.95%
Appendix B: Training Details	18	≈ 2.56 pp.	11.64%
Appendix C: Additional Empirical Analyses	21	≈ 1.6 pp.	7.27%

All detected figure and table locations

Main text

• Table 1 · p. 2
• Figure 1 · p. 3
• Table 2 · p. 6
• Table 3 · p. 6
• Table 4 · p. 6
• Table 5 · p. 6
• Table 6 · p. 7
• Figure 2 · p. 7
• Figure 3 · p. 7
• Figure 4 · p. 8
• Figure 5 · p. 8
• Table 7 · p. 9

Attached appendix

• Table 8 · p. 20
• Table 9 · p. 20
• Table 10 · p. 20
• Figure 6 · p. 21
• Table 11 · p. 21
• Table 12 · p. 22
• Figure 7 · p. 22

Argument and organization

1. Problem and contribution

   Uses one associative-memory equation and a parameter table to locate several existing SSMs before naming geometry, gradient and interference concerns.

2. Argument sequence

   The update is followed by separate parallelizability, gradient and rank claims; experiments then distinguish downstream accuracy from capacity and normalization controls.

3. Evidence and verification

   A doubled-state baseline challenges the capacity explanation; a three-way momentum/Frobenius/Newton–Schulz control pairs effective rank with validation accuracy.

Problem, claim and contribution

Uses one associative-memory equation and a parameter table to locate several existing SSMs before naming geometry, gradient and interference concerns.

Table 1 and Section 2, p. 2 · Main paper

Argument and section order

The update is followed by separate parallelizability, gradient and rank claims; experiments then distinguish downstream accuracy from capacity and normalization controls.

Sections 2-5, pp. 2-8 · Main paper

Figures, captions and visual explanation

The baseline mapping table exposes algebraic differences rather than only model names. Matched gradient heatmaps share a color scale, with the exact norm and axes defined beneath them.

Table 1, p. 2; Figure 7 and Table 12, p. 22; both PNGs inspected · Main paper

Tables, comparisons and evidence

A doubled-state baseline challenges the capacity explanation; a three-way momentum/Frobenius/Newton–Schulz control pairs effective rank with validation accuracy.

Section 5.3, pp. 7-8; Table 12, p. 22 · Main paper

Appendix and reproducibility

The proof preserves rank-one individual injections, while accumulation raises attainable rank; the final diagnostic explicitly says nonlinear heatmaps are not a strict non-vanishing-gradient guarantee.

Appendix A.3-A.5, pp. 15-18; C.2, p. 22 · Main paper

Clarity and writing conventions

The strongest qualification sits beside the supporting diagnostic, distinguishing a linear structural route from its empirical behavior in a nonlinear network.

Appendix C.2, p. 22 · Main paper

What to reuse—and what not to infer

Transferable lesson: Use a shared algebraic baseline map and targeted controls to separate added capacity from the proposed update geometry.

Boundary: Selected GradCAM examples do not establish a general shape bias, and empirical gradient preservation should not be promoted into an unconditional deep-network theorem.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-focused distributed manuscript scan, not exhaustive technical/proof validation. Sections 1-7, pp. 1-9: unified memory update, momentum/normalization, scan and geometry analysis, multimodal evaluation and mechanism controls.

Appendix / supplement

Writing-focused appendix scan; code, data and external assets not executed. Appendices A-C, pp. 13-22: proposition-specific proofs, training implementation and modality settings, normalization/effective-rank ablation and gradient heatmaps.

Visually inspected PDF pages

2, 22

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published ICML 2026 proceedings PDF, PMLR volume 306; retrieved 2026-10-07. Public OpenReview decision notes were not independently reverified. · 22 pages