Writing study: On the Difficulty of Learning a Meta-network for Training Data Selection

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Zilin Du, Junqi Zhao, Boyang Li · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Empirical analysis, diagnosis & auditing (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

45 physical pages

Before references / appendix

≈ 9.14 page-equivalents

Main figures / tables

3 / 4

Attached appendix figures / tables

2 / 2

Attached appendix span

≈ 31.91 page-equivalents

Before references / appendix: ≈ 20.31%References / intervening material: ≈ 8.78%Attached appendix: ≈ 70.91%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Source-bound correction: Include the actual appendix outline on physical p.14 in the supporting-material envelope; proof section A itself begins p.15. The outline entries are not section starts. Newly detected unnumbered impact/singular-acknowledgment statements are retained from the size-aware source extraction; authored heading corrections are unchanged.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 1.24 pp.	2.76%
2. Related Work	2	≈ 0.82 pp.	1.82%
3. Difficulty in Training Selection Network	3	≈ 3.21 pp.	7.13%
4. Input Features for the Selection Network	6	≈ 0.56 pp.	1.24%
5. Experiments	7	≈ 2.62 pp.	5.82%
6. Conclusion	9	≈ 0.13 pp.	0.29%
Impact Statement	9	≈ 0.16 pp.	0.36%
Acknowledgment	10	≈ 0.08 pp.	0.18%
References	10	≈ 3.94 pp.	8.76%
Appendix Outline	14	≈ 1 pp.	2.22%
A. Derivation of the Hypergradient Expressions	15	≈ 2.25 pp.	5%
B. Theorem 3.2 on the Dynamics of Normalized Data Weights pi	17	≈ 6.75 pp.	15%
C. Theorem 3.4 and Corollary 3.5 on the Sensitivity of ∆ϕ to wK	24	≈ 13.15 pp.	29.22%
D. Theorem 3.6 on the Upper Bound of GSNR	37	≈ 3.57 pp.	7.93%
E. Training Dynamics across Batch Sizes 512 and 1024	40	≈ 0.58 pp.	1.29%
F. Details of the Datasets	41	≈ 0.34 pp.	0.76%
G. Details of the Baselines	41	≈ 1.49 pp.	3.31%
H. Additional Experimental Details	43	≈ 0.44 pp.	0.98%
I. Details of the Input Features	43	≈ 1.08 pp.	2.4%
J. Algorithm Details	44	≈ 1.26 pp.	2.8%

All detected figure and table locations

Main text

• Figure 1 · p. 2
• Figure 2 · p. 5
• Table 1 · p. 8
• Table 2 · p. 8
• Table 3 · p. 9
• Table 4 · p. 9
• Figure 3 · p. 9

Attached appendix

• Figure 2 · p. 40
• Figure 4 · p. 41
• Table 5 · p. 42
• Table 6 · p. 43

Argument and organization

1. Problem and claim

   Defines the bilevel data-weighting problem before naming two obstacles: weak gradient signal and uninformative selector features. This creates a reason for testing batch size and inputs separately.

2. Argument sequence

   Moves from weight dynamics to hypergradient sensitivity and GSNR, then to practical features and experiments. The theory-to-intervention transition interprets the bound terms instead of presenting a theorem without a design consequence.

3. Evidence and verification

   The experiment section separates few-shot validation, backbone transfer and a cost trade-off. Figure 3 encodes normalized training time, accuracy and memory together; the largest-batch result visibly requires much more memory.

Problem, claim and contribution

Defines the bilevel data-weighting problem before naming two obstacles: weak gradient signal and uninformative selector features. This creates a reason for testing batch size and inputs separately.

Abstract and Introduction, pp. 1–2 · Main paper

Argument and section order

Moves from weight dynamics to hypergradient sensitivity and GSNR, then to practical features and experiments. The theory-to-intervention transition interprets the bound terms instead of presenting a theorem without a design consequence.

Sections 3–5, pp. 3–8 · Main paper

Figures, captions and visual explanation

Figure 2 repeats four diagnostic panels for two batch sizes, reusing dataset colors and defining effective batch size in the caption. Shading denotes standard deviation across independent batches, not seed confidence intervals.

Figure 2, physical p. 5; raster inspected · Main paper

Tables, comparisons and evidence

The experiment section separates few-shot validation, backbone transfer and a cost trade-off. Figure 3 encodes normalized training time, accuracy and memory together; the largest-batch result visibly requires much more memory.

Tables 3–4 and Figure 3, physical p. 9; raster inspected · Main paper

Appendix and reproducibility

A dedicated contents page maps lemma/theorem derivations to named sections, followed by datasets, baselines, implementation, features and pseudocode. This makes the long mathematical evidence record navigable without interrupting the main explanation.

Appendix outline p. 14; Appendices A–J, pp. 15–45 · Main paper

Clarity and writing conventions

The weight-dynamics discussion distinguishes independent-weight theory from selector-network behavior and describes the observed pattern as qualitatively consistent. The transfer experiment should remain evidence on the tested backbones, not a universal architecture-independent guarantee.

Section 3.3, pp. 4–5; Section 5.4, p. 9 · Main paper

What to reuse—and what not to infer

Transferable lesson: Connect each theoretical factor to a diagnostic and an intervention, then display the resource cost of the resulting practical remedy.

Boundary: A larger batch is not a free improvement, and smooth selector-network dynamics need not realize the limiting one-hot behavior of the simplified analysis.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-focused scan of Sections 1–6, pp. 1–9, including formal assumptions, linked training-dynamics plots, feature controls and accuracy–memory comparisons. No independent verification of theorems.

Appendix / supplement

Structural scan of the appendix outline and Appendices A–J, pp. 14–45; selected proof assumptions, dataset/baseline descriptions, implementation and algorithm details checked. Long derivations were not audited line by line.

Visually inspected PDF pages

5, 9

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published proceedings PDF, PMLR volume 306, retrieved 2026-10-07 · 45 pages