Writing study: On the Convergence Rate of LoRA Gradient Descent

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Siqiao Mu, Diego Klabjan · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Theory & formal analysis (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

19 physical pages

Before references / appendix

≈ 7.73 page-equivalents

Main figures / tables

5 / 0

Attached appendix figures / tables

3 / 1

Attached appendix span

≈ 7.91 page-equivalents

Before references / appendix: ≈ 40.68%References / intervening material: ≈ 17.68%Attached appendix: ≈ 41.63%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 1.29 pp.	6.79%
2. Related Work	2	≈ 0.75 pp.	3.95%
3. Convergence Analysis	3	≈ 3.01 pp.	15.84%
4. Experiments	6	≈ 2 pp.	10.53%
5. Conclusion	8	≈ 0.3 pp.	1.58%
Impact Statement	8	≈ 0.05 pp.	0.26%
References	8	≈ 3.35 pp.	17.63%
A. Proofs	12	≈ 6.22 pp.	32.74%
B. Experimental Details	18	≈ 1.69 pp.	8.89%

All detected figure and table locations

Main text

• Figure 1 · p. 6
• Figure 2 · p. 7
• Figure 3 · p. 7
• Figure 4 · p. 8
• Figure 5 · p. 8

Attached appendix

• Table 1 · p. 18
• Figure 6 · p. 18
• Figure 7 · p. 19
• Figure 8 · p. 19

Argument and organization

1. Problem and contribution

   Identifies which prior LoRA analyses change the update or assume bounded adapters, then states the original-loss assumptions retained here.

2. Argument sequence

   Original and reparameterized losses are defined separately before descent lemmas; the main rate follows a step-size-sum argument and leads to practical schedules.

3. Evidence and verification

   The theorem bounds the minimum squared reparameterized gradient norm and gives a separate bounded-iterate rate; stochastic-gradient analysis remains future work.

Problem, claim and contribution

Identifies which prior LoRA analyses change the update or assume bounded adapters, then states the original-loss assumptions retained here.

Introduction and related work, pp. 2-3 · Main paper

Argument and section order

Original and reparameterized losses are defined separately before descent lemmas; the main rate follows a step-size-sum argument and leads to practical schedules.

Sections 3.1-3.3, pp. 3-6 · Main paper

Figures, captions and visual explanation

The formulation page gives simultaneous A/B updates explicit equations. Aligned experiment panels expose loss, parameter norm and gradient norm, while the LLM caption discloses moving-average smoothing.

Equations 1-5, p. 3; Figures 4-5, p. 8; both PNGs inspected · Main paper

Tables, comparisons and evidence

The theorem bounds the minimum squared reparameterized gradient norm and gives a separate bounded-iterate rate; stochastic-gradient analysis remains future work.

Theorem 3.5, p. 5; conclusion, p. 8 · Main paper

Appendix and reproducibility

Proof subsections map directly to main lemmas and theorem; extra full-rank comparisons specify different smoothing windows for logistic regression and ResNet.

Appendix A, pp. 12-18; Figures 7-8, p. 19 · Main paper

Clarity and writing conventions

The empirical discussion distinguishes the schedule closest to theory from variants more convenient in practice.

Section 4 continuation, p. 8 · Main paper

What to reuse—and what not to infer

Transferable lesson: Name the exact convergence quantity and assumptions before summarizing a rate, and label practical departures from the analyzed algorithm.

Boundary: A minimum-gradient stationarity rate is not a global-optimum, last-iterate or accuracy guarantee; mini-batch experiments do not establish the deferred stochastic theorem.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-focused distributed manuscript scan, not exhaustive technical/proof validation. Sections 1-5, pp. 1-8: assumption gap, stacked-parameter analysis, adaptive step-size theorem, interpretation and practical experiments.

Appendix / supplement

Writing-focused appendix scan; code, data and external assets not executed. Appendices A-B, pp. 12-19: helper lemmas, lemma/theorem proofs, setup and full-rank comparison plots.

Visually inspected PDF pages

3, 8

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published ICML 2026 proceedings PDF, PMLR volume 306; retrieved 2026-10-07. Public OpenReview decision notes were not independently reverified. · 19 pages