Writing study: Universal Inverse Distillation for Matching Models with Real-Data Supervision (No GANs)

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Nikita Kornilov, David Li, Tikhon Mavrin, Aleksei Leonov, Nikita Gushchin, Evgeny Burnaev, Iaroslav Koshelev, Aleksandr Korotin · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Learning & optimization algorithms (primary) · Theory & formal foundations.

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

43 physical pages

Before references / appendix

≈ 10.35 page-equivalents

Main figures / tables

2 / 4

Attached appendix figures / tables

13 / 4

Attached appendix span

≈ 28.65 page-equivalents

Before references / appendix: ≈ 24.07%References / intervening material: ≈ 9.3%Attached appendix: ≈ 66.63%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 0.8 pp.	1.86%
2 BACKGROUNDS ON TRAINING AND DISTILLING MATCHING MODELS	2	≈ 2.29 pp.	5.33%
3 UNIVERSAL DISTILLATION OF MATCHING MODELS WITH REAL DATA	4	≈ 3.59 pp.	8.35%
4 EXPERIMENTS	8	≈ 2.31 pp.	5.37%
5 DISCUSSION AND EXTENSIONS	10	≈ 0.38 pp.	0.88%
6 BROADER IMPACT	11	≈ 0.15 pp.	0.35%
7 LLM USAGE	11	≈ 0.09 pp.	0.21%
REFERENCES	11	≈ 4 pp.	9.3%
A THEORETICAL PROOFS AND EXTENSIONS	15	≈ 12.76 pp.	29.67%
B REALUID ALGORITHM FOR FLOW MATCHING MODELS	28	≈ 1 pp.	2.33%
C UNIFIED INVERSE DISTILLATION WITH REAL DATA FOR BRIDGE	29	≈ 2 pp.	4.65%
D EXPERIMENTAL DETAILS AND ADDITIONAL RESULTS	31	≈ 12.89 pp.	29.98%

All detected figure and table locations

Main text

• Figure 1 · p. 6
• Table 1 · p. 9
• Table 2 · p. 9
• Figure 2 · p. 9
• Table 3 · p. 10
• Table 4 · p. 10

Attached appendix

• Figure 3 · p. 18
• Figure 4 · p. 20
• Figure 5 · p. 24
• Table 5 · p. 32
• Table 6 · p. 32
• Table 7 · p. 33
• Table 8 · p. 34
• Figure 6 · p. 34
• Figure 7 · p. 35
• Figure 8 · p. 36
• Figure 9 · p. 37
• Figure 10 · p. 38
• Figure 11 · p. 39
• Figure 12 · p. 40
• Figure 13 · p. 41
• Figure 14 · p. 42
• Figure 15 · p. 43

Argument and organization

1. Background and common matching loss

   Normalize diffusion/flow notation before comparing distillation objectives.

2. Universal inverse distillation and real data

   Expose an intractable term, linearize it, connect prior methods, and introduce real-data weighting.

3. Unified experiments

   Test the theory-informed coefficient choice before broader benchmark comparisons.

4. Theoretical and experimental appendices

   Separate extensions, negative cases, implementation algorithm, hyperparameters, and sample galleries.

Problem, claim and contribution

Two explicit gaps—fragmented distillation formulations and adversarial machinery for incorporating real data—map directly onto the two contributions. The background establishes shared notation before claiming universality.

Sections 1–2, pages 1–4. · Main paper

Argument and section order

The derivation marks the intractable expectation, introduces a single linearization identity, and then interprets the resulting objective as inverse optimization. Short summary and coefficient-choice paragraphs keep a dense proof-led narrative navigable.

Sections 3.1–3.4, pages 5–8. · Main paper

Figures, captions and visual explanation

Figure 1 consistently assigns red to generator, green to fake model, and blue to frozen teacher; actual images distinguish real/generated inputs. Its caption also states alternating update frequency. Small formulas merit a zoomable version.

Figure 1, page 6. · Main paper

Tables, comparisons and evidence

The ablation table displays the full coefficient grid, including inferior configurations and infeasible entries, before presenting fine-tuned winners. Curves mark the fine-tuning boundary and teacher/GAN references, connecting mechanism to training behavior.

Tables 1–2 and Figure 2, page 9. · Main paper

Appendix and reproducibility

A detailed contents page leads to proofs, a runnable-style alternating-update algorithm, implementation settings, and uncurated galleries. The unsuccessful heuristic-SiD extension is retained rather than hidden.

Page 14; Appendix A.3, pages 22–24; Appendices B and D, pages 28–43. · Main paper

Clarity and writing conventions

The main comparison explicitly concedes lower FID performance than some adversarial diffusion methods, preserving the narrower contribution. However, red/green-only cell semantics and very small Table 2 text reduce accessibility.

Section 4.3, page 10; Tables 1–2, page 9. · Main paper

What to reuse—and what not to infer

Transferable lesson: For a unification paper, establish common notation, identify the precise mathematical obstacle, and test a theory-derived hyperparameter prediction under fixed architecture before citing heterogeneous leaderboard results.

Boundary: The title's 'No GANs' does not mean no min-max optimization: a learned fake matching model remains. Arbitrary teacher-error correction is not unconditional; Appendix A.1.4 explicitly assumes a zero-attainable distance for its intuition. Main empirical evidence is low-resolution image generation; bridge/coupling extensions are theoretical, and Appendix A.3 reports instability for heuristic SiD settings.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

Pages 1–11, Sections 1–7; reference list excluded from analytical reading.

Appendix / supplement

Pages 14–43, contents and Appendices A–D in full, including proofs, Algorithm 1, experimental protocols, and sample captions. Visual sample grids were not exhaustively inspected.

Visually inspected PDF pages

6, 9

• Official ICLR 2026 proceedings conference PDF; PDF creation metadata: Mon May 11 10:34:18 2026 EDT · 43 pages