Writing study: Scaling Laws and Spectra of Shallow Neural Networks in the Feature Learning Regime

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Leonardo Defilippis, Yizhou Xu, Julius Girardin, Vittorio Erba, Emanuele Troiani, Lenka Zdeborova, Bruno Loureiro, Florent Krzakala · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Theory & formal foundations (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

42 physical pages

Before references / appendix

≈ 10.11 page-equivalents

Main figures / tables

3 / 0

Attached appendix figures / tables

5 / 0

Attached appendix span

≈ 27.29 page-equivalents

Before references / appendix: ≈ 24.07%References / intervening material: ≈ 10.95%Attached appendix: ≈ 64.98%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 3.23 pp.	7.69%
2 MAIN RESULTS	4	≈ 5.71 pp.	13.6%
3 CONCLUSION	10	≈ 0.28 pp.	0.67%
ACKNOWLEDGEMENTS	10	≈ 0.34 pp.	0.81%
REFERENCES	11	≈ 4.61 pp.	10.98%
A THE BRIDGE FROM SPARSE ESTIMATION TO NEURAL NETWORKS	15	≈ 1.64 pp.	3.9%
B NOISELESS SETTING: ERROR RATES AND SPECTRAL PROPERTIES	17	≈ 1.51 pp.	3.6%
C DERIVATION DETAILS - DIAGONAL LINEAR NETWORK	18	≈ 12.33 pp.	29.36%
D DERIVATION DETAILS - QUADRATIC NEURAL NETWORKS	31	≈ 11.81 pp.	28.12%

All detected figure and table locations

Main text

• Figure 1 · p. 3
• Figure 2 · p. 7
• Figure 3 · p. 10

Attached appendix

• Figure 4 · p. 18
• Figure 5 · p. 18
• Figure 6 · p. 28
• Figure 7 · p. 28
• Figure 8 · p. 29

Argument and organization

1. Two model reductions and contributions

   Connects diagonal and quadratic networks to LASSO and matrix sensing before stating what the analysis explains.

2. Rates, spectra and interpretation

   Uses one effective-sample-size convention to organize a phase diagram, spectral predictions and an error decomposition.

3. Conjecture, numerical checks and derivations

   Separates the non-asymptotic assumption from empirical support and places model-specific calculations in the appendix.

Problem, claim and contribution

The introduction narrows the broad scaling-law problem to two solvable architectures with Gaussian inputs and power-law target weights. The reductions make the scope concrete before the phase diagram appears.

Sections 1 and 1.1, pages 1–3. · Main paper

Argument and section order

Numbered results progress from rates to spectra to interpretable error terms; Section 2.4 then explicitly labels the extension beyond proportional asymptotics a conjecture.

Results 1–3, pages 5–7; Conjecture 1, page 9. · Main paper

Figures, captions and visual explanation

Figure 1 aligns colored risk regimes, boundary formulas and miniature spectral sketches on the same axes. It is a compact navigation map, although the equations require substantial statistical background.

Figure 1, page 6. · Main paper

Tables, comparisons and evidence

The spectral grid distinguishes simulation histograms from bulk/spike predictions and discloses the omitted zero mass. The learning-curve grid separates architectures and regularization, overlaying numerical dots, theory curves and slope guides.

Figure 2 and caption, page 7; Figure 3 and caption, page 10. · Main paper

Appendix and reproducibility

Derivations are split by architecture and noisy/noiseless setting. Numerical details disclose a failed approximation, its Monte Carlo replacement, the optimizer, code location and substantial CPU/memory costs.

Appendices C–F, pages 18–42; implementation and resources on page 42. · Main paper

Clarity and writing conventions

Underbraces label the decomposition as learned noise, unlearned features and error in learned features, turning a dense equation into a conceptual explanation. The following phase-by-phase prose reuses those terms consistently.

Equations 17–18, page 7; Section 2.3, pages 8–9. · Main paper

What to reuse—and what not to infer

Transferable lesson: Use a single phase diagram as the paper's organizing map, connect mathematical terms to interpretable phenomena, and place conjectural assumptions beside the numerical evidence that supports—but does not prove—them.

Boundary: The analysis concerns global minimizers of specific shallow models, not general deep-network training or compute scaling. Non-asymptotic state evolution and the sub-extensive-rank simplification are not fully proved; numerical agreement does not remove those qualifications.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

Pages 1–10, Sections 1–3; references on pages 11–15.

Appendix / supplement

Pages 15–42, Appendices A–F: equivalences, noiseless regimes, diagonal and quadratic derivations, ridge comparison and numerical implementation.

Visually inspected PDF pages

6, 7, 10, 42

• ICLR 2026 official proceedings PDF, retrieved 2026-09-19 · 42 pages