Writing study: Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Arnas Uselis, Andrea Dittadi, Seong Joon Oh · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Theory & formal analysis (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

68 physical pages

Before references / appendix

≈ 9.21 page-equivalents

Main figures / tables

10 / 0

Attached appendix figures / tables

35 / 3

Attached appendix span

≈ 53.92 page-equivalents

Before references / appendix: ≈ 13.54%References / intervening material: ≈ 7.16%Attached appendix: ≈ 79.29%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Source-bound correction: Include the actual appendix cover/contents page in the attached supporting-material envelope, before its first lettered section. Contents entries are not counted as section starts. Newly detected unnumbered impact/singular-acknowledgment statements are retained from the size-aware source extraction; authored heading corrections are unchanged.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 1.03 pp.	1.51%
2. Related work	2	≈ 0.43 pp.	0.63%
3. Setup: A framework for compositionality	2	≈ 2.5 pp.	3.68%
4. Implications of compositionality on	5	≈ 1.45 pp.	2.13%
5. Surveying necessary conditions in	6	≈ 2.96 pp.	4.35%
6. Discussion and conclusion	9	≈ 0.35 pp.	0.51%
Acknowledgments	10	≈ 0.11 pp.	0.16%
Impact statement	10	≈ 0.05 pp.	0.07%
References	10	≈ 4.87 pp.	7.16%
Appendix — contents	15	≈ 1 pp.	1.47%
A. Extended related work	16	≈ 2 pp.	2.94%
B. Necessary conditions (proof of Proposition 1)	18	≈ 5 pp.	7.35%
C. Sufficiency of linear factorization for compositional generalization	23	≈ 6 pp.	8.82%
D. Packing and minimum dimension (proof of Proposition 2)	29	≈ 2 pp.	2.94%
E. Examples of linearly compositional models and their geometries	31	≈ 6 pp.	8.82%
F. Discussion on stability	37	≈ 1 pp.	1.47%
G. Additional information	38	≈ 4.3 pp.	6.32%
H. Additional experimental results	42	≈ 26.61 pp.	39.13%

All detected figure and table locations

Main text

• Figure 1 · p. 1
• Figure 2 · p. 2
• Figure 3 · p. 3
• Figure 4 · p. 5
• Figure 5 · p. 5
• Figure 6 · p. 6
• Figure 7 · p. 7
• Figure 8 · p. 8
• Figure 9 · p. 8
• Figure 10 · p. 9

Attached appendix

• Figure 11 · p. 19
• Figure 12 · p. 27
• Figure 13 · p. 29
• Figure 14 · p. 32
• Figure 15 · p. 33
• Figure 16 · p. 37
• Figure 17 · p. 38
• Figure 18 · p. 39
• Figure 19 · p. 41
• Table 1 · p. 42
• Figure 21 · p. 42
• Table 2 · p. 43
• Figure 20 · p. 44
• Figure 22 · p. 47
• Figure 23 · p. 47
• Figure 24 · p. 48
• Figure 25 · p. 48
• Figure 26 · p. 49
• Figure 27 · p. 51
• Table 3 · p. 52
• Figure 28 · p. 53
• Figure 29 · p. 53
• Figure 30 · p. 54
• Figure 31 · p. 55
• Figure 32 · p. 56
• Figure 33 · p. 56
• Figure 34 · p. 57
• Figure 35 · p. 58
• Figure 36 · p. 59
• Figure 37 · p. 60
• Figure 38 · p. 60
• Figure 39 · p. 61
• Figure 40 · p. 62
• Figure 41 · p. 63
• Figure 42 · p. 64
• Figure 43 · p. 66
• Figure 44 · p. 67
• Figure 45 · p. 68

Argument and organization

1. 1 Introduction; 2 Related work; 3 Setup

   Separates necessary from sufficient conditions and specifies concepts, valid supports, readouts and three desiderata.

2. 4 Implications of compositionality on representations

   States conditional geometric and dimensional results with proof sketches and takeaway boxes.

3. 5 Surveying necessary conditions; 6 Discussion and conclusion

   Tests partial factorization and orthogonality in pretrained models, then states gaps between ideal stability and practical training.

Problem, claim and contribution

The claim is necessity under a stated learning framework, not that every representation in any intelligent system must be exactly additive.

Sections 2–4, physical pp.2–5 · Main paper

Argument and section order

The formal objects and training-support rules precede the propositions; empirical diagnostics are introduced only after the necessary-condition argument.

Sections 3–5, pp.3–8 · Main paper

Figures, captions and visual explanation

The large geometric explanation moves from a global PCA view to conditioned concepts, additive factors and pairwise projections; separate scatterplots quantify the proposed association.

Figure 7, p.7; Figures 8–9, p.8 · Main paper

Tables, comparisons and evidence

Projected, whitened R-squared is distinguished from generalization accuracy. The text acknowledges partial rather than perfect factorization and reports a random-model baseline.

Sections 5.1–5.2 and Figures 8–9, p.8 · Main paper

Appendix and reproducibility

A counterexample section explains why linear separability alone does not force the claimed geometry; the metric appendix shows why dominant directions can inflate unwhitened R-squared.

Appendix F and Figure 16, p.37; G.1–G.2 and Figure 17, p.38 · Main paper

Clarity and writing conventions

The conclusion names worst-case stability and the fixed-encoder/retrained-readout assumption, providing a concrete boundary for applying the result.

Section 6, p.9 · Main paper

What to reuse—and what not to infer

Transferable lesson: State the model class, data-support family and stability requirement before a necessity claim, then distinguish an empirical proxy from the formal condition.

Boundary: The main binary-grid proposition assumes linear heads, gradient descent with cross-entropy and specific valid-support families. Correlation of whitened factorization with accuracy does not independently prove causal necessity in practical trained encoders. Proofs were not independently checked.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-focused scan across physical PDF pp.1–9, covering framing, section progression, definitions, method/evidence explanations, captions and scope. Physical pages extracted individually to avoid form-feed pagination errors. Selected full-page rasters inspected; not exhaustive line-by-line technical verification or experimental reproduction.

Appendix / supplement

Appendix heading inventory through H.7 (p.65), with selected detail reading of Stability counterexamples F (p.37) and the projection/whitening procedure G.1–G.2 (p.38). Long proofs and the complete experimental appendix were not independently verified.

Visually inspected PDF pages

7, 8

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published proceedings PDF, PMLR volume 306, retrieved 2026-10-07 · 68 pages