Writing study: The Polar Express: Optimal Matrix Sign Methods and their Application to the Muon Algorithm

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Noah Amsel, David Persson, Christopher Musco, Robert M. Gower · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Learning & optimization algorithms (primary) · Theory & formal foundations · Inference, algorithms & systems efficiency.

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

38 physical pages

Before references / appendix

≈ 10.23 page-equivalents

Main figures / tables

6 / 0

Attached appendix figures / tables

12 / 0

Attached appendix span

≈ 22.89 page-equivalents

Before references / appendix: ≈ 26.92%References / intervening material: ≈ 12.84%Attached appendix: ≈ 60.24%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 2.78 pp.	7.32%
2 APPROXIMATIONS BY COMPOSITIONS OF POLYNOMIALS	4	≈ 1.09 pp.	2.87%
3 THE POLAR EXPRESS	5	≈ 3.05 pp.	8.03%
4 NUMERICAL EXPERIMENTS	8	≈ 2.65 pp.	6.97%
REFERENCES	11	≈ 4.88 pp.	12.84%
A CODE FOR P O L A R E X P R E S S	16	≈ 1.5 pp.	3.95%
B RELATED WORK	17	≈ 2.1 pp.	5.53%
C PROOF OF THEOREM 3.1	19	≈ 3.02 pp.	7.95%
D PROOF OF THEOREM 3.3	22	≈ 0.61 pp.	1.61%
E PROOF OF EQUIVALENCE BETWEEN (5) AND (6)	23	≈ 0.36 pp.	0.95%
F REMEZ ALGORITHM	23	≈ 2.42 pp.	6.37%
G FINITE PRECISION CONSIDERATIONS	26	≈ 0.65 pp.	1.71%
H ADDITIONAL EXPERIMENTAL RESULTS	26	≈ 3.96 pp.	10.42%
I INITIALIZATION FOR MATRICES WITH LARGE SPECTRAL GAPS	30	≈ 4.97 pp.	13.08%
J FAST POLYNOMIAL ITERATION FOR RECTANGULAR MATRICES	35	≈ 3.31 pp.	8.71%

All detected figure and table locations

Main text

• Figure 1 · p. 2
• Figure 2 · p. 5
• Figure 3 · p. 8
• Figure 4 · p. 9
• Figure 5 · p. 10
• Figure 6 · p. 10

Attached appendix

• Figure 7 · p. 27
• Figure 8 · p. 27
• Figure 9 · p. 29
• Figure 10 · p. 29
• Figure 11 · p. 31
• Figure 12 · p. 32
• Figure 13 · p. 33
• Figure 14 · p. 34
• Figure 15 · p. 34
• Figure 16 · p. 34
• Figure 17 · p. 36
• Figure 18 · p. 37

Argument and organization

1. Application-specific numerical requirements

   Explain why classical high-accuracy solvers do not fit GPU optimizer needs.

2. Scalar minimax reduction and greedy construction

   Reduce matrix approximation to a polynomial problem and state the exact optimality class.

3. Finite precision and measured optimizer effects

   Separate ideal theory from practical stabilization and downstream training evidence.

Problem, claim and contribution

The paper motivates a numerical-linear-algebra contribution through throughput and precision requirements, explicitly holding Muon's update rule fixed. This keeps the algorithmic claim narrower than inventing a new optimizer.

Sections 1–1.3, pages 1–4 · Main paper

Argument and section order

The derivation moves from matrix spectral error to a scalar uniform approximation and then a greedy composition theorem. Each reduction supplies a reason why the proposed implementation is computationally appropriate.

Sections 2–3.2, pages 4–6 · Main paper

Figures, captions and visual explanation

Figure 2 visualizes successive interval compression with consistently shaded ranges and the target line at one, turning the equioscillation argument into a readable sequence before the theorem box.

Figure 2 and Theorem 3.1, page 5 · Main paper

Tables, comparisons and evidence

The iteration ablation pairs final loss with step time and includes an exact-SVD reference. It demonstrates that more accurate polar factors need not improve model training, avoiding a proxy-metric-only argument.

Figure 5, page 10; Appendix H.1, pages 26–29 · Main paper

Appendix and reproducibility

A contents map routes readers to complete code, proofs and finite-precision modifications. The practical implementation includes stabilization choices beyond the ideal theorem; these should be identified as such when reused.

Appendix A, pages 16–17; Appendices C–G, pages 19–26 · Main paper

Clarity and writing conventions

The supplementary image-classification experiments openly report no consistent advantage, and the rectangular-matrix extension reports no end-to-end runtime gain at GPT-Small scale. These negative findings bound the positive language-model narrative.

Appendix H.3, page 30; Appendix J.1, pages 37–38 · Main paper

What to reuse—and what not to infer

Transferable lesson: For a numerical subroutine, connect the exact theorem's admissible algorithm class to a concrete implementation, then test whether better numerical accuracy actually improves the downstream task.

Boundary: Minimax optimality concerns compositions of fixed-degree odd polynomials over a bounded spectrum in the ideal analysis, not universal optimizer optimality. Finite-precision cushioning changes that ideal method; vision gains are not consistently positive.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

Pages 1–11, Sections 1–4.3 and reproducibility statement

Appendix / supplement

Pages 15–38, contents and Appendices A–J.1, implementation, proofs, numerical stability and all extra experiments

Visually inspected PDF pages

5, 10, 16

• ICLR 2026 final proceedings · 38 pages