Writing study: Neon: Negative Extrapolation From Self-Training Improves Image Generation

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

sina alemohammad, Zhangyang Wang, Richard Baraniuk · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Learning & optimization algorithms (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

57 physical pages

Before references / appendix

≈ 10.1 page-equivalents

Main figures / tables

10 / 0

Attached appendix figures / tables

23 / 2

Attached appendix span

≈ 41.9 page-equivalents

Before references / appendix: ≈ 17.72%References / intervening material: ≈ 8.77%Attached appendix: ≈ 73.51%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Source-bound correction: Visually confirmed Figure 8 on page 9: caption label occupies its own Poppler line and the caption wording begins on the next line. Figures 9 and 10 are separate real figures, not references. Caption audit 2026-09-19, kpLRYtPGt3: rendered physical PDF page 9 confirms Figure 8, the cross-architecture synthetic-data-transfer line chart. Its bare Figure 8: label is separated from caption text in bbox lines. Position uses yMin 255.306558 / 792 plus page index 8. Appendix alphanumeric labels are retained as printed, including K.0; no relabeling is inferred.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	2	≈ 0.75 pp.	1.32%
2 BACKGROUND	2	≈ 0.99 pp.	1.74%
3 NEON: NEGATIVE EXTRAPOLATION FROM SELF-TRAINING	3	≈ 2.73 pp.	4.79%
4 EXPERIMENTS	6	≈ 3.58 pp.	6.28%
5 CONCLUSIONS	10	≈ 0.35 pp.	0.61%
ACKNOWLEDGMENTS	10	≈ 0.14 pp.	0.25%
ETHICS STATEMENT	10	≈ 0.46 pp.	0.81%
REFERENCES	11	≈ 5 pp.	8.77%
A STATE OF THE ART COMPARISON	16	≈ 2 pp.	3.51%
B PROOFS AND DETAILED EXPLANATIONS	18	≈ 21 pp.	36.84%
C EXPERIMENTS DETAILS	39	≈ 3 pp.	5.26%
D ADDITIONAL EXPERIMENTS FOR DIFFUSION AND FLOW MATCHING	42	≈ 1 pp.	1.75%
E XAR-B ON IMAGENET-256 SYNTHESIZED IMAGES	43	≈ 1 pp.	1.75%
F XAR-L ON IMAGENET-256 SYNTHESIZED IMAGES	44	≈ 1 pp.	1.75%
G VAR-D16 ON IMAGENET-256 SYNTHESIZED IMAGES	45	≈ 1 pp.	1.75%
H IMM ON IMAGENET-256 SYNTHESIZED IMAGES	46	≈ 4 pp.	7.02%
I VAR-D36-S ON IMAGENET-512 SYNTHESIZED IMAGES	50	≈ 4 pp.	7.02%
J EDM-VP ON FFHQ-64 SYNTHESIZED IMAGES	54	≈ 1 pp.	1.75%
K VISUALIZING NEON’S EFFECT ON INDIVIDUAL SAMPLES	55	≈ 2.9 pp.	5.09%

All detected figure and table locations

Main text

• Figure 1 · p. 1
• Figure 2 · p. 4
• Figure 3 · p. 6
• Figure 4 · p. 7
• Figure 5 · p. 7
• Figure 6 · p. 8
• Figure 7 · p. 8
• Figure 8 · p. 9
• Figure 9 · p. 9
• Figure 10 · p. 9

Attached appendix

• Table A.1 · p. 16
• Figure B.1 · p. 20
• Table B.1 · p. 23
• Figure B.2 · p. 37
• Figure B.3 · p. 38
• Figure D.1 · p. 42
• Figure E.1 · p. 43
• Figure F.1 · p. 44
• Figure G.1 · p. 45
• Figure H.1 · p. 46
• Figure H.2 · p. 47
• Figure H.3 · p. 47
• Figure H.4 · p. 48
• Figure H.5 · p. 48
• Figure H.6 · p. 49
• Figure H.7 · p. 49
• Figure I.1 · p. 50
• Figure I.2 · p. 51
• Figure I.3 · p. 51
• Figure I.4 · p. 52
• Figure I.5 · p. 52
• Figure I.6 · p. 53
• Figure I.7 · p. 53
• Figure J.1 · p. 54
• Figure K.0 · p. 57

Argument and organization

1. Reverse a short self-training update

   Introduces an inexpensive parameter-space intervention before explaining its proposed mechanism.

2. Local gradient geometry

   States conditions intended to distinguish beneficial extrapolation from harmful overstepping.

3. Families, trade-offs and implementation details

   Tests diffusion, flow and autoregressive models while exposing tuning and sampler dependencies.

Problem, claim and contribution

The paper turns self-training degradation into a constructive question: can reversing the update help? The one-line merge keeps the intervention memorable while distinguishing training-time adaptation from extra inference-time guidance or model compression.

§§1–3, pp.1–5 · Main paper

Argument and section order

The method, local explanation and model-family experiments form a clear progression. AppendixB adds a Gaussian warmup and proof roadmap, but its general guarantees remain conditional on near-optimum geometry and sampler assumptions rather than following from the merge identity alone.

§3, pp.3–5; AppendicesB.1–B.3, pp.18–23 · Main paper

Figures, captions and visual explanation

Aligned FID, precision and recall panels reveal the price of improvement: recall can improve while precision falls. The shared merge-weight axis is more informative than a single best-score table; appendix grids additionally expose interaction with classifier-free guidance.

Figure4, p.7; FigureD.1, p.42; FiguresI.2–I.7, pp.51–53 · Main paper

Tables, comparisons and evidence

ImageNet256 xAR-L FID improves from1.28 to1.02. AppendixC discloses10k-sample hyperparameter selection followed by50k-sample reporting, including joint guidance search; training images seen are a budget proxy, not a complete measured compute bill.

§4.2 and Figure5, p.7; TableA.1c, p.16; AppendixC, pp.39–41 · Main paper

Appendix and reproducibility

Buffer handling, normalization and stable merge arithmetic are valuable reproducibility details. However, the512-resolution model changes between VAR-d30 and VAR-d36-s labels, and the supplied theorem proof changes bias-term signs between LemmaB.3 and TheoremB.4.

AppendixB.5, pp.25–26; AppendixC, pp.40–41; AppendixI, p.50 · Main paper

Clarity and writing conventions

Visual proofreading catches errors text extraction misses: FigureH.5 labels car grids as a container ship and FigureI.7 labels gorillas as cheeseburgers. Captions are part of the scientific claim and require the same consistency checks as table numbers.

FigureH.5, p.48; FigureI.7, p.53 · Main paper

What to reuse—and what not to infer

Transferable lesson: Pair a simple intervention with trade-off plots, disclose the entire selection protocol, and audit equations, captions and model identifiers across the main paper and appendix before publication.

Boundary: No-real-data fine-tuning still uses real reference statistics for FID selection. Theoretical sign and monotonicity arguments need independent verification; acceptance and complete reading do not certify them. Lower FID is not uniformly better precision, no-overhead training, or evidence of pathology transfer.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

pp.1–10, §§1–6; references pp.11–15

Appendix / supplement

pp.16–57, A–K: complete comparison table, all supplied theoretical derivations, toy experiment, configurations, trade-off curves and image galleries

Visually inspected PDF pages

7, 26, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57

• official ICLR 2026 proceedings PDF · 57 pages