Writing study: DiScoFormer: Plug-In Density and Score Estimation with Transformers

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Vasily Ilin, Peter Sushko, Ranjay Krishna · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Architecture & representation (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

21 physical pages

Before references / appendix

≈ 10.04 page-equivalents

Main figures / tables

11 / 5

Attached appendix figures / tables

4 / 0

Attached appendix span

≈ 7.91 page-equivalents

Before references / appendix: ≈ 47.81%References / intervening material: ≈ 14.52%Attached appendix: ≈ 37.67%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 0.97 pp.	4.62%
2. Related Work	2	≈ 0.95 pp.	4.52%
3. Methodology	3	≈ 2.52 pp.	12%
4. Experiments	5	≈ 2.82 pp.	13.43%
5. Conclusion	8	≈ 0.27 pp.	1.29%
Impact Statement	8	≈ 2.18 pp.	10.38%
References	11	≈ 3.04 pp.	14.48%
A. Proofs	14	≈ 3.16 pp.	15.05%
B. Supplementary Theoretical Results	17	≈ 1.53 pp.	7.29%
C. Attention visualization	18	≈ 1.77 pp.	8.43%
D. Comparison with the Score Matching Loss	20	≈ 0.27 pp.	1.29%
E. Runtime comparison	20	≈ 1.18 pp.	5.62%

All detected figure and table locations

Main text

• Table 1 · p. 3
• Figure 1 · p. 3
• Figure 2 · p. 4
• Figure 3 · p. 6
• Figure 4 · p. 6
• Table 2 · p. 7
• Table 3 · p. 7
• Figure 5 · p. 7
• Table 4 · p. 7
• Table 5 · p. 7
• Figure 6 · p. 8
• Figure 7 · p. 9
• Figure 8 · p. 9
• Figure 9 · p. 9
• Figure 10 · p. 9
• Figure 11 · p. 9

Attached appendix

• Figure 12 · p. 18
• Figure 13 · p. 19
• Figure 14 · p. 20
• Figure 15 · p. 21

Argument and organization

1. Problem and contribution

   Defines one operator that accepts a context sample and query points, contrasting amortized inference with distribution-specific model training.

2. Argument sequence

   Equivariance definitions and constructive representation arguments precede synthetic training and applications; limitations distinguish the training family from untested target distributions.

3. Evidence and verification

   Label-free consistency adaptation is distinct from supervised synthetic pretraining. The conclusion separates exact translation/scaling behavior from approximate rotation behavior and leaves asymptotic consistency unproven.

Problem, claim and contribution

Defines one operator that accepts a context sample and query points, contrasting amortized inference with distribution-specific model training.

Abstract and Introduction, p. 1; Section 3, pp. 3-5 · Main paper

Argument and section order

Equivariance definitions and constructive representation arguments precede synthetic training and applications; limitations distinguish the training family from untested target distributions.

Sections 3-5, pp. 3-8 · Main paper

Figures, captions and visual explanation

The runtime page uses paired dimension-specific log-log panels, with solid measured curves and dashed fitted slopes; its caption preserves a baseline out-of-memory endpoint and the shared quadratic scaling.

Figure 15, p. 21; PNG inspected; opening contract p. 1 also visually inspected · Main paper

Tables, comparisons and evidence

Label-free consistency adaptation is distinct from supervised synthetic pretraining. The conclusion separates exact translation/scaling behavior from approximate rotation behavior and leaves asymptotic consistency unproven.

Sections 3.4-3.5, p. 5; Section 5, p. 8 · Main paper

Appendix and reproducibility

The exact KDE construction explicitly assumes a bare attention layer, exact arithmetic and a squared-norm lift; runtime analysis separately compares a modern attention implementation to naive KDE.

Appendix A, pp. 14-16; E, pp. 20-21 · Main paper

Clarity and writing conventions

A mathematical interface appears before the list of applications, giving the varied density, entropy and physics examples a common object of comparison.

Introduction, p. 1; Sections 4.1-4.7, pp. 6-8 · Main paper

What to reuse—and what not to infer

Transferable lesson: Separate representational possibility, learned empirical behavior and implementation-specific speed within the same architecture argument.

Boundary: An exact attention construction is not proof that learned weights implement KDE, and a naive-baseline runtime comparison is not an algorithm-independent speed claim.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-focused distributed manuscript scan, not exhaustive technical/proof validation. Sections 1-5, pp. 1-10: equivariant-operator framing, architecture and synthetic training distribution, diverse estimation tasks, limitations and main figures.

Appendix / supplement

Writing-focused appendix scan; code, data and external assets not executed. Appendices A-E, pp. 14-21: constructive proofs, whitening/generalization arguments, attention analysis, score-matching comparison and runtime scaling.

Visually inspected PDF pages

1, 21

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published ICML 2026 proceedings PDF, PMLR volume 306; retrieved 2026-10-07. Public OpenReview decision notes were not independently reverified. · 21 pages