Writing study: How much can language models memorize?

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

John Xavier Morris, Chawin Sitawarin, Chuan Guo, Narine Kokhlikyan, G. Edward Suh, Alexander M Rush, Kamalika Chaudhuri, Saeed Mahloujifar · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Empirical analysis, diagnosis & auditing (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

20 physical pages

Before references / appendix

≈ 8.89 page-equivalents

Main figures / tables

11 / 1

Attached appendix figures / tables

9 / 4

Attached appendix span

≈ 8.46 page-equivalents

Before references / appendix: ≈ 44.45%References / intervening material: ≈ 13.25%Attached appendix: ≈ 42.3%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Source-bound correction: Source-checked physical-page text restores wrapped top-level heading labels on pp. 6. Heading starts and measured positions are unchanged; this is label completeness, not a new section. Newly detected unnumbered impact/singular-acknowledgment statements are retained from the size-aware source extraction; authored heading corrections are unchanged.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 1.02 pp.	5.1%
2. Memorization, intended and unintended	2	≈ 2.07 pp.	10.35%
3. Model Capacity for Memorization	4	≈ 2.24 pp.	11.2%
4. Disentangling Unintended Memorization from Generalization	6	≈ 1.12 pp.	5.6%
5. Memorization and Membership	7	≈ 1.93 pp.	9.65%
6. Conclusion	9	≈ 0.1 pp.	0.5%
Impact Statement	9	≈ 0.11 pp.	0.55%
References	9	≈ 2.65 pp.	13.25%
A. Appendix	12	≈ 8.46 pp.	42.3%

All detected figure and table locations

Main text

• Figure 1 · p. 2
• Figure 2 · p. 2
• Figure 11 · p. 5
• Figure 3 · p. 5
• Figure 4 · p. 5
• Table 1 · p. 6
• Figure 5 · p. 6
• Figure 15 · p. 7
• Figure 6 · p. 8
• Figure 8 · p. 8
• Figure 7 · p. 8
• Figure 9 · p. 9

Attached appendix

• Figure 10 · p. 13
• Figure 11 · p. 13
• Table 2 · p. 15
• Table 4 · p. 15
• Table 3 · p. 15
• Figure 12 · p. 16
• Figure 13 · p. 16
• Figure 14 · p. 17
• Figure 15 · p. 17
• Figure 16 · p. 17
• Table 5 · p. 18
• Figure 17 · p. 20
• Figure 18 · p. 20

Argument and organization

1. Problem and contribution

   Three explicit requirements separate unintended memorization from generalization, demand sample-level interpretation, and avoid dependence of the definition on an inaccessible training algorithm.

2. Argument sequence

   A local roadmap moves from distributional Shannon information to individual-string Kolmogorov complexity and then practical likelihood estimates before any capacity experiment.

3. Evidence and verification

   Uniform synthetic data provides known entropy and a no-generalization case; five-seed sweeps, convergence checks and a precision comparison support an observed capacity lower bound rather than an optimization-independent maximum.

Problem, claim and contribution

Three explicit requirements separate unintended memorization from generalization, demand sample-level interpretation, and avoid dependence of the definition on an inaccessible training algorithm.

Section 2, p. 2 · Main paper

Argument and section order

A local roadmap moves from distributional Shannon information to individual-string Kolmogorov complexity and then practical likelihood estimates before any capacity experiment.

Section 2 roadmap and Sections 2.1-3, pp. 2-4 · Main paper

Figures, captions and visual explanation

Matched train/test encodings organize loss, extraction and membership plots; paired compression histograms show why average counts can conceal different sample-level distributions.

Figures 3-4, p. 5; Figures 6-8, p. 8; both PNGs inspected · Main paper

Tables, comparisons and evidence

Uniform synthetic data provides known entropy and a no-generalization case; five-seed sweeps, convergence checks and a precision comparison support an observed capacity lower bound rather than an optimization-independent maximum.

Section 3.2, p. 5; Appendix A.5, p. 15 · Main paper

Appendix and reproducibility

A same-distribution reference-size ablation documents disjoint data, reuses target runs, and shows that absolute memorization estimates change with the reference while qualitative trends persist.

Appendix A.13 and Figures 17-18, pp. 18-20 · Main paper

Clarity and writing conventions

The paper explicitly labels exact Kolmogorov complexity uncomputable and states how likelihoods approximate it; the final limitations restrict transfer across architectures, datasets and training setups.

Section 2.3, p. 4; Appendix A.14, p. 19 · Main paper

What to reuse—and what not to infer

Transferable lesson: Introduce a measurement through requirements, ideal definition and operational approximation, then perturb its reference choices before interpreting absolute values.

Boundary: Estimated bits are reference-relative; empirical plateaus are lower bounds under tested optimization, and average membership/extraction trends do not constitute privacy guarantees for individual records.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-focused distributed manuscript scan, not exhaustive technical/proof validation. Sections 1-6, pp. 1-9: memorization requirements, information-theoretic definitions, likelihood approximations, synthetic capacity, real-text behavior and membership/extraction scaling.

Appendix / supplement

Writing-focused appendix scan; code, data and external assets not executed. Appendix A.1-A.14, pp. 12-20: alternative definitions, compression details, training convergence, content analysis, fitting/proofs, reference-model sensitivity and limitations.

Visually inspected PDF pages

5, 8

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published ICML 2026 proceedings PDF, PMLR volume 306; retrieved 2026-10-07. Public OpenReview decision notes were not independently reverified. · 20 pages