Writing study: How Many Different Outputs Can a Transformer Generate?

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Maxime Meyer, Mario Michelessa, Caroline Chaux, Vincent Tan · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Theory & formal analysis (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

30 physical pages

Before references / appendix

≈ 9.54 page-equivalents

Main figures / tables

5 / 1

Attached appendix figures / tables

7 / 2

Attached appendix span

≈ 16.91 page-equivalents

Before references / appendix: ≈ 31.8%References / intervening material: ≈ 11.83%Attached appendix: ≈ 56.37%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Source-bound correction: Source-checked physical-page text restores wrapped top-level heading labels on pp. 5. Heading starts and measured positions are unchanged; this is label completeness, not a new section. Newly detected unnumbered impact/singular-acknowledgment statements are retained from the size-aware source extraction; authored heading corrections are unchanged.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 1.96 pp.	6.53%
2. The Transformer Architecture	3	≈ 0.94 pp.	3.13%
3. Representing the Embedding Space	4	≈ 0.84 pp.	2.8%
4. The Sequences that are Accessible to a Transformer	5	≈ 1.7 pp.	5.67%
5. Experiments	6	≈ 2.9 pp.	9.67%
6. Conclusion	9	≈ 0.13 pp.	0.43%
7. Future Work	9	≈ 0.26 pp.	0.87%
Impact Statement	10	≈ 0.05 pp.	0.17%
Acknowledgements	10	≈ 0.45 pp.	1.5%
References	10	≈ 3.54 pp.	11.8%
A. Transformer Layers	14	≈ 0.66 pp.	2.2%
B. Mean-Field Transformers	14	≈ 1.1 pp.	3.67%
C. Packing Number	15	≈ 2.24 pp.	7.47%
D. Permutation-Invariant Norms and Wasserstein Distances	18	≈ 1 pp.	3.33%
E. Arbitrary Prompt Length via Elementary Operations	19	≈ 1.4 pp.	4.67%
F. Theoretical Slope of the Cramming Task for Various Supports	20	≈ 3 pp.	10%
G. Supplementary Experimental Results	23	≈ 1.96 pp.	6.53%
H. Computing the Cell Volume Distribution	25	≈ 5.56 pp.	18.53%

All detected figure and table locations

Main text

• Figure 1 · p. 4
• Figure 2 · p. 7
• Table 1 · p. 8
• Figure 2c · p. 8
• Figure 3 · p. 8
• Figure 4 · p. 9

Attached appendix

• Figure 5 · p. 15
• Figure 6 · p. 22
• Table 2 · p. 23
• Table 3 · p. 26
• Figure 7 · p. 27
• Figure 8 · p. 28
• Figure 9 · p. 29
• Figure 10 · p. 29
• Figure 11 · p. 30

Argument and organization

1. Problem and contribution

   Defines accessibility through existence of a prompt producing a sequence under greedy decoding, distinguishing finite-prompt capacity from an arbitrary-length mean-field setting.

2. Argument sequence

   Decoder partitions lead to packing bounds, then the empirical section tests threshold scaling before relaxing geometric approximations and studying copying generalization.

3. Evidence and verification

   Even refined ratios remain above one, and future work explicitly leaves direct empirical evaluation of the arbitrary-prompt theorem and lower bounds unresolved.

Problem, claim and contribution

Defines accessibility through existence of a prompt producing a sequence under greedy decoding, distinguishing finite-prompt capacity from an arbitrary-length mean-field setting.

Definition 4.1 and Sections 4.1-4.2, pp. 5-6 · Main paper

Argument and section order

Decoder partitions lead to packing bounds, then the empirical section tests threshold scaling before relaxing geometric approximations and studying copying generalization.

Sections 3-5, pp. 4-9 · Main paper

Figures, captions and visual explanation

The theorem page puts uniform-grid assumptions and a not-all-long-sequences caveat next to the bound; the refinement table reports theory/empirical ratios rather than implying a tight prediction.

Theorem 4.6/Remarks 4.5 and 4.8, p. 5; Table 1/Figure 3, p. 8; both PNGs inspected · Main paper

Tables, comparisons and evidence

Even refined ratios remain above one, and future work explicitly leaves direct empirical evaluation of the arbitrary-prompt theorem and lower bounds unresolved.

Table 1, p. 8; Section 7, p. 9 · Main paper

Appendix and reproducibility

Separate derivations handle ball/cone/ellipsoid supports and non-uniform precision, followed by model-specific support and decoder-cell distributions.

Appendices C/F-H, pp. 15-30 · Main paper

Clarity and writing conventions

Named assumptions distinguish an intuitive finite grid from the stronger Wasserstein-precision condition, while the narrative's broad capacity language needs those qualifications carried forward.

Assumption 4.3/Remark 4.5, p. 5; Assumption 4.9/Remark 4.12, p. 6 · Main paper

What to reuse—and what not to infer

Transferable lesson: State the computational and geometric assumptions before an expressivity bound, then show the remaining theory–measurement gap and the untested theorem separately.

Boundary: Failure of optimized soft prompts or length-generalization copying is not proof that no prompt exists; these greedy-decoding, finite-precision assumptions do not bound arbitrary stochastic generation in the same way.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-focused distributed manuscript scan, not exhaustive technical/proof validation. Sections 1-7, pp. 1-9: output accessibility, decoder-cell geometry, finite-prompt and mean-field bounds, cramming tests, support refinements and copying-length generalization.

Appendix / supplement

Writing-focused appendix scan; code, data and external assets not executed. Appendices A-H, pp. 14-30: transformer/mean-field definitions, packing proofs, precision assumptions, elementary-operation alternative, support geometry and empirical cell-volume calculations.

Visually inspected PDF pages

5, 8

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published ICML 2026 proceedings PDF, PMLR volume 306; retrieved 2026-10-07. Public OpenReview decision notes were not independently reverified. · 30 pages