Writing study: Overparametrization bends the landscape: BBP transitions at initialization in simple Neural Networks

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Brandon Annesi, Dario Bocchi, Chiara Cammarota · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Theory & formal foundations (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

26 physical pages

Before references / appendix

≈ 10.25 page-equivalents

Main figures / tables

4 / 0

Attached appendix figures / tables

6 / 0

Attached appendix span

≈ 13.59 page-equivalents

Before references / appendix: ≈ 39.42%References / intervening material: ≈ 8.31%Attached appendix: ≈ 52.27%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 1.72 pp.	6.62%
2 THE MODEL	3	≈ 2.03 pp.	7.81%
3 METHODOLOGICAL OVERVIEW	5	≈ 0.79 pp.	3.04%
4 RESULTS	6	≈ 3.44 pp.	13.23%
5 CONCLUSION AND DISCUSSION	9	≈ 1.63 pp.	6.27%
REFERENCES	11	≈ 2.16 pp.	8.31%
A FIELD THEORY APPROACH	13	≈ 6.69 pp.	25.73%
B ANALYTICAL COMPUTATIONS FOR THE BBP	20	≈ 2.18 pp.	8.38%
C DERIVATION OF THE OVERLAP	22	≈ 4.72 pp.	18.15%

All detected figure and table locations

Main text

• Figure 1 · p. 6
• Figure 2 · p. 7
• Figure 3 · p. 8
• Figure 4 · p. 9

Attached appendix

• Figure 5 · p. 21
• Figure 6 · p. 23
• Figure 7 · p. 24
• Figure 8 · p. 24
• Figure 9 · p. 25
• Figure 10 · p. 26

Argument and organization

1. Introduction / The model

   Motivates the geometry question, then fixes the teacher–student setting and measurable overlap.

2. Methodological overview

   Teaches the diagrammatic calculation on the smallest case before the general appendix derivation.

3. Results / Conclusion and discussion

   Separates asymptotic phase transitions, finite-size observations and remaining dynamical questions.

4. Appendices A–F

   Provides derivations and scoped extensions rather than duplicating the result narrative.

Problem, claim and contribution

The broad optimization motivation narrows to curvature at random initialization, not a theorem about entire training trajectories. Defining the sample ratio, widths and normalized loss prevents an informal claim about all overparameterized networks.

§§1–2, pp.1–5 · Main paper

Argument and section order

The methodological overview uses the single-node setting and one worked diagram before sending full block-matrix calculations to Appendix A. Readers can grasp the analytical instrument before encountering the general result.

§3, pp.5–6; Appendix A, pp.13–19 · Main paper

Figures, captions and visual explanation

Figure 2 links a global threshold-versus-normalization plot to three fixed-parameter insets; colors distinguish continuous and discontinuous regimes. Figure 3 compares finite dimensions on aligned axes to expose disagreement with the asymptotic transition.

Figures 2–3, pp.7–8 · Main paper

Tables, comparisons and evidence

The infinite-width result states its order of limits explicitly. Finite-size threshold extrapolation is labeled a conjecture, with its fit shown, rather than promoted into the same evidence class as the analytical expression.

Eq.18, p.7; §4.2 and Figure 4, pp.8–9 · Main paper

Appendix and reproducibility

Appendices D–F vary teacher width, examine sigmoid/tanh activations and connect to gradient descent. The general-activation derivation is explicitly beyond scope and the dynamics experiment explicitly preliminary.

Appendices D–F, pp.23–26 · Main paper

Clarity and writing conventions

Bold discussion lead-ins distinguish qualitative transition changes, exceptions to beneficial overparameterization, the large-width limit and finite-size effects. Exceptions remain visible instead of disappearing into a universally positive conclusion.

§5, pp.9–10 · Main paper

What to reuse—and what not to infer

Transferable lesson: Teach a difficult proof technique on a minimal example, distinguish asymptotic mathematics from finite-system evidence, and keep counterexamples adjacent to the main trend.

Boundary: Initialization signal recovery is not a full gradient-descent guarantee. The conjectured finite-size threshold and qualitative non-quadratic extensions must retain their weaker status; reading the derivation is not independent theorem verification.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

PDF pp.1–10, abstract and §§1–5; references pp.11–13

Appendix / supplement

PDF pp.13–26, Appendices A–F and LLM disclosure, including field-theory derivations, general activations and preliminary dynamics

Visually inspected PDF pages

7, 8

• official ICLR 2026 proceedings PDF · 26 pages