Writing study: FIRE: Frobenius-Isometry Reinitialization for Balancing the Stability–Plasticity Tradeoff

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Isaac Han, Sangyeon Park, Seungwon Oh, Donghu Kim, Hojoon Lee, KyungJoong Kim · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Learning & optimization algorithms (primary) · Theory & formal foundations.

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

33 physical pages

Before references / appendix

≈ 10.26 page-equivalents

Main figures / tables

5 / 0

Attached appendix figures / tables

3 / 10

Attached appendix span

≈ 17.89 page-equivalents

Before references / appendix: ≈ 31.09%References / intervening material: ≈ 14.7%Attached appendix: ≈ 54.21%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 1.18 pp.	3.58%
2 RELATED WORK	2	≈ 0.8 pp.	2.42%
3 METHOD	3	≈ 2.93 pp.	8.88%
4 EXPERIMENTS	6	≈ 4.1 pp.	12.42%
5 CONCLUSION	10	≈ 0.23 pp.	0.7%
REPRODUCIBILITY STATEMENT	10	≈ 0.25 pp.	0.76%
ACKNOWLEDGMENTS	11	≈ 0.16 pp.	0.48%
REFERENCES	11	≈ 4.84 pp.	14.67%
A PROOF OF THEOREMS	16	≈ 7 pp.	21.21%
B ADDITIONAL RESULTS	23	≈ 2 pp.	6.06%
C IMPLEMENTATION DETAILS OF FIRE	25	≈ 1 pp.	3.03%
D BASELINE METHODS	26	≈ 2 pp.	6.06%
E DETAILED EXPERIMENT SETTINGS	28	≈ 5.89 pp.	17.85%

All detected figure and table locations

Main text

• Figure 1 · p. 1
• Figure 2 · p. 7
• Figure 3 · p. 8
• Figure 4 · p. 9
• Figure 5 · p. 10

Attached appendix

• Table 1 · p. 23
• Figure 6 · p. 23
• Figure 7 · p. 24
• Figure 8 · p. 24
• Table 2 · p. 28
• Table 3 · p. 28
• Table 4 · p. 29
• Table 5 · p. 29
• Table 6 · p. 30
• Table 7 · p. 31
• Table 8 · p. 32
• Table 9 · p. 32
• Table 10 · p. 33

Argument and organization

1. Two measurable desiderata

   Defines weight proximity and deviation from isometry before posing a constrained reset.

2. Theory to a short iteration

   Links proxies to conditional bounds and then gives Newton–Schulz pseudocode.

3. Three learning regimes

   Separates visual continual learning, small-language-model pretraining and reinforcement learning.

Problem, claim and contribution

The paper explicitly distinguishes continual adaptation with old data available from restricted-memory catastrophic-forgetting settings. This scope statement prevents an otherwise ambiguous stability claim.

Introduction page 2; limitations page 10 · Main paper

Argument and section order

Separate subsections define stability and plasticity proxies before the constrained problem, making the polar projection a consequence of stated criteria rather than a disconnected trick.

Sections 3.1–3.3, pages 3–6 · Main paper

Figures, captions and visual explanation

The short PyTorch-like Algorithm 1 is placed beside the approximation explanation; this is an effective way to make a matrix-based method approachable after several theorem statements.

Algorithm 1 and Section 3.4, page 6 · Main paper

Tables, comparisons and evidence

Learning curves emphasize both the reset discontinuity and subsequent recovery, which matches the stability–plasticity thesis better than final accuracy alone.

Figures 2–5, pages 7–10 · Main paper

Appendix and reproducibility

Appendices disclose layer-specific scope, scaling factors, seed counts and search spaces. They reveal that ViT resets affect Q/K projections rather than every weight matrix.

Appendices C–E, pages 25–33 · Main paper

Clarity and writing conventions

The core iteration should be consistent across text and supplement: the main algorithm uses a cubic polynomial, while Appendix B.3 says the adopted iteration is a different quintic polynomial.

Algorithm 1 page 6 versus Appendix B.3 pages 23–24 · Main paper

What to reuse—and what not to infer

Transferable lesson: Turn an intuitive tradeoff into explicit proxies, state the regime where those proxies are useful, and show intervention-time behavior rather than relying only on final scores.

Boundary: The theory bounds depend on whitening, activation and rank assumptions and do not make orthogonality universally optimal. Main and appendix Newton–Schulz coefficients disagree, so reproduction needs implementation confirmation.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

Pages 1–10, Sections 1–5 and reproducibility statement

Appendix / supplement

Pages 16–33, Appendices A–E including all proofs, implementation choices and hyperparameter tables

Visually inspected PDF pages

6, 23

• ICLR 2026 final proceedings · 33 pages