Writing study: Non-Euclidean Gradient Descent Operates at the Edge of Stability

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Rustem Islamov, Michael Crawshaw, Jeremy Cohen, Robert M. Gower · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Theory & formal analysis (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

27 physical pages

Before references / appendix

≈ 8.75 page-equivalents

Main figures / tables

6 / 0

Attached appendix figures / tables

13 / 0

Attached appendix span

≈ 15.91 page-equivalents

Before references / appendix: ≈ 32.41%References / intervening material: ≈ 8.67%Attached appendix: ≈ 58.93%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 1.73 pp.	6.41%
2. Progressive Sharpening and Directional	3	≈ 1.34 pp.	4.96%
3. Examples of Non-Euclidean Gradient	4	≈ 2.91 pp.	10.78%
4. Normalized Non-Euclidean Gradient	7	≈ 0.52 pp.	1.93%
5. Towards Understanding the Underlying	7	≈ 1.5 pp.	5.56%
6. Conclusion and Future Work	9	≈ 0.33 pp.	1.22%
Impact Statement	9	≈ 0.08 pp.	0.3%
References	9	≈ 2.34 pp.	8.67%
A. Discussion on Frank-Wolfe Algorithm	12	≈ 0.54 pp.	2%
B. An oscillatory regime before EoS	12	≈ 0.96 pp.	3.56%
C. The gap between the generalized sharpness and 2/η	13	≈ 0.67 pp.	2.48%
D. Training Details	14	≈ 0.13 pp.	0.48%
E. Additional Experimental Results with ℓ∞ Descent	14	≈ 1.33 pp.	4.93%
F. Additional Experimental Results with Block Gradient Descent	15	≈ 1.13 pp.	4.19%
G. Additional Experimental Results with Spectral Gradient Descent	16	≈ 2.26 pp.	8.37%
H. ℓ∞-descent and RMSprop	19	≈ 0.6 pp.	2.22%
I. Useful Lemmas	19	≈ 4.4 pp.	16.3%
J. Non-Euclidean Gradient Descent on Quadratics	24	≈ 3.91 pp.	14.48%

All detected figure and table locations

Main text

• Figure 1 · p. 4
• Figure 2 · p. 6
• Figure 3 · p. 6
• Figure 4 · p. 7
• Figure 5 · p. 8
• Figure 6 · p. 8

Attached appendix

• Figure 7 · p. 13
• Figure 8 · p. 13
• Figure 9 · p. 14
• Figure 10 · p. 15
• Figure 11 · p. 15
• Figure 12 · p. 16
• Figure 13 · p. 16
• Figure 14 · p. 16
• Figure 15 · p. 17
• Figure 16 · p. 17
• Figure 17 · p. 18
• Figure 18 · p. 18
• Figure 19 · p. 19

Argument and organization

1. 1 Introduction; 2 Progressive Sharpening and Directional Smoothness

   Defines the update geometry and links loss change to a directional quantity before introducing generalized sharpness.

2. 3 Examples; 4 Normalized Non-Euclidean Gradient Descent

   Recovers known cases and compares new norms using repeated diagnostic panels.

3. 5 Towards Understanding the Underlying Mechanism; 6 Conclusion and Future Work

   Uses quadratic stability results and a Taylor-switch experiment, then separates established facts from open mechanisms.

Problem, claim and contribution

The contribution is a geometry-aware diagnostic, not a claim that a new optimizer improves test performance.

Sections 1–2, physical pp.1–3 · Main paper

Argument and section order

Definitions, familiar special cases, new geometries and a restricted mechanistic result form an explicit progression.

Sections 2–5, pp.3–8 · Main paper

Figures, captions and visual explanation

Repeated columns align loss, gradient norm, directional smoothness and generalized sharpness; the later intervention switches from the real objective to a local quadratic approximation.

Figures 2–3, p.6; Figures 5–6, p.8 · Main paper

Tables, comparisons and evidence

Experiments are full-batch and sharpness can be an approximate nonconvex optimization result. The main paper distinguishes observed hovering from exact identities and conditional quadratic theorems.

Section 3, p.4; Theorems 5.1–5.2, pp.8–9 · Main paper

Appendix and reproducibility

Approximation discussion says stationarity is not global optimality; restart sensitivity and precise full-batch training defaults support interpretation of the plotted diagnostic.

Appendix A, p.12; D–E, p.14 · Main paper

Clarity and writing conventions

The conclusion explicitly states that divergence above the threshold is constructed for a particular initialization, not proved generic for every non-Euclidean trajectory.

Section 5 continuation and Section 6, p.9 · Main paper

What to reuse—and what not to infer

Transferable lesson: Use consistent diagnostic panels across cases, then distinguish an exact identity, a heuristic estimate, a theorem under assumptions and an empirical mechanism test.

Boundary: This is full-batch evidence, not a guarantee for stochastic, momentum-based or adaptive large-scale training. Frank–Wolfe estimates can understate generalized sharpness, and the divergence construction does not prove generic divergence. Formal proofs were not independently verified.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-focused scan across physical PDF pp.1–9, covering framing, section progression, definitions, method/evidence explanations, captions and scope. Physical pages extracted individually to avoid form-feed pagination errors. Selected full-page rasters inspected; not exhaustive line-by-line technical verification or experimental reproduction.

Appendix / supplement

Appendix heading inventory A–J checked; selected detailed reading of Frank–Wolfe approximation and pre-EoS oscillations A–B (p.12), and training/estimator sensitivity D–E (p.14). Full norm-specific derivations and quadratic proofs were not independently verified.

Visually inspected PDF pages

6, 8

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published proceedings PDF, PMLR volume 306, retrieved 2026-10-07 · 27 pages