Writing study: Prescriptive Scaling Reveals the Evolution of Language Model Capabilities

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Hanlin Zhang, Jikai Jin, Vasilis Syrgkanis, Sham M. Kakade · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Empirical analysis, diagnosis & auditing (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

31 physical pages

Before references / appendix

≈ 8.69 page-equivalents

Main figures / tables

7 / 3

Attached appendix figures / tables

22 / 1

Attached appendix span

≈ 17.91 page-equivalents

Before references / appendix: ≈ 28.03%References / intervening material: ≈ 14.19%Attached appendix: ≈ 57.77%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Extraction notes: Table numbering discontinuity; see source note. Zero means no recognized caption, not independently verified absence.

Source-bound correction: Source-checked physical-page text restores wrapped top-level heading labels on pp. 2, 4, 6, 7. Heading starts and measured positions are unchanged; this is label completeness, not a new section. Newly detected unnumbered impact/singular-acknowledgment statements are retained from the size-aware source extraction; authored heading corrections are unchanged.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 1.1 pp.	3.55%
2. Estimation of Post-training Capability Boundaries	2	≈ 1.36 pp.	4.39%
3. Sigmoid Scaling Laws for Post-training Performance Boundaries	4	≈ 2.18 pp.	7.03%
4. Capability Boundary Estimation under Limited Budget	6	≈ 0.98 pp.	3.16%
5. Case Studies: Saturation and Contamination Diagnostics	7	≈ 1.52 pp.	4.9%
6. Related Works	8	≈ 0.53 pp.	1.71%
7. Conclusion	9	≈ 0.26 pp.	0.84%
Acknowledgment	9	≈ 0.1 pp.	0.32%
Impact Statement	9	≈ 0.05 pp.	0.16%
References	9	≈ 4.39 pp.	14.16%
A. Pinball Loss and Quantile Regression	14	≈ 1.61 pp.	5.19%
B. Additional Details for Section 2.1–2.3	15	≈ 0.75 pp.	2.42%
C. Pre-training vs. Post-training Diagnostics	16	≈ 0.39 pp.	1.26%
D. Details and Additional Analyses for Section 3	16	≈ 3.56 pp.	11.48%
E. Scaling Laws for Model Size	20	≈ 0.23 pp.	0.74%
F. Newly Evaluated models	20	≈ 2.02 pp.	6.52%
G. Sensitivity to Smoothed-Pinball Hyperparameters	22	≈ 3.13 pp.	10.1%
H. Additional Results	25	≈ 2.73 pp.	8.81%
I. Saturation Analysis across Open LLM Leaderboard Versions and Tasks	28	≈ 0.25 pp.	0.81%
J. Greedy Optimization for the Balanced I-Optimal Design	28	≈ 3.24 pp.	10.45%

All detected figure and table locations

Main text

• Table 1 · p. 2
• Figure 1 · p. 4
• Table 2 · p. 4
• Figure 2 · p. 4
• Figure 3 · p. 5
• Figure 4 · p. 5
• Figure 5 · p. 6
• Figure 6 · p. 7
• Table 3 · p. 8
• Figure 7 · p. 8

Attached appendix

• Figure 8 · p. 14
• Figure 9 · p. 17
• Figure 10 · p. 17
• Figure 11 · p. 18
• Figure 12 · p. 19
• Figure 13 · p. 19
• Figure 14 · p. 20
• Table 5 · p. 20
• Figure 15 · p. 21
• Figure 16 · p. 21
• Figure 17 · p. 22
• Figure 18 · p. 23
• Figure 19 · p. 24
• Figure 20 · p. 25
• Figure 21 · p. 25
• Figure 22 · p. 26
• Figure 23 · p. 26
• Figure 24 · p. 27
• Figure 25 · p. 27
• Figure 26 · p. 28
• Figure 27 · p. 29
• Figure 28 · p. 31
• Figure 29 · p. 31

Argument and organization

1. Problem and contribution

   Narrows a broad prescriptive training question to an empirical 0.98 conditional quantile in the observed model ecosystem, explicitly not a true universal maximum or a claim that compute is the sole cause.

2. Argument sequence

   Defines the estimand, estimator families and complementary calibration metrics before chronological tests, external families, budget reduction and interpretive case studies.

3. Evidence and verification

   Constant, binwise and flexible monotone baselines test model complexity, while temporal splits and newly evaluated families challenge stability; the AIME analysis reports n=90, p=0.15 and leaves modest effects unresolved.

Problem, claim and contribution

Narrows a broad prescriptive training question to an empirical 0.98 conditional quantile in the observed model ecosystem, explicitly not a true universal maximum or a claim that compute is the sole cause.

Sections 2-2.1, pp. 2-3 · Main paper

Argument and section order

Defines the estimand, estimator families and complementary calibration metrics before chronological tests, external families, budget reduction and interpretive case studies.

Sections 2-5, pp. 3-8 · Main paper

Figures, captions and visual explanation

Consistent train/validation colors distinguish fitted curves from observed points; frontier-model plots sit next to text reporting the nonsignificant contamination diagnostic rather than implying absence from visual overlap alone.

Figures 3-4, p. 5; Figure 7, p. 8; both PNGs inspected · Main paper

Tables, comparisons and evidence

Constant, binwise and flexible monotone baselines test model complexity, while temporal splits and newly evaluated families challenge stability; the AIME analysis reports n=90, p=0.15 and leaves modest effects unresolved.

Section 2.2, p. 3; Sections 3.3 and 5.2, pp. 6, 8 · Main paper

Appendix and reproducibility

Bin-level coverage reveals signs and locations hidden by scalar loss; the selection algorithm states parameter-count cost, anchor initialization and optional exchange refinement rather than promising exact budget-optimal selection.

Appendix D.2, pp. 19-20; J, pp. 28-31 · Main paper

Clarity and writing conventions

Remarks explain how a new recipe can move the estimated boundary, and the appendix distinguishes observational saturation plots from a controlled parameter-scaling law.

Remark 3.1, p. 5; Appendix I, p. 28 · Main paper

What to reuse—and what not to infer

Transferable lesson: State the estimand before the fitted curve, challenge it across time and model families, and pair aggregate error with location-specific calibration.

Boundary: A high conditional quantile is not a per-model success guarantee; pretraining FLOPs omit heterogeneous post-training cost, parameter-weighted evaluation budget is not elapsed time, and a nonsignificant shift does not establish uncontaminated benchmarks.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-focused distributed manuscript scan, not exhaustive technical/proof validation. Sections 1-7, pp. 1-9: empirical capability-envelope definition, quantile estimators and diagnostics, chronological validation, budgeted selection and saturation/contamination case studies.

Appendix / supplement

Writing-focused appendix scan; code, data and external assets not executed. Appendices A-J, pp. 14-31: loss intuition, binning and spline construction, temporal and family diagnostics, sensitivity, extended leaderboards and greedy-selection implementation.

Visually inspected PDF pages

5, 8

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published ICML 2026 proceedings PDF, PMLR volume 306; retrieved 2026-10-07. Public OpenReview decision notes were not independently reverified. · 31 pages