Skip to content
Computational Pathology Reading Roadmap

Writing study: Transformers are Inherently Succinct

← ICLR 2026 oral writing corpus

Pascal Bergsträßer, Ryan Cotterell, Anthony W. Lin · Official ICLR 2026 oral record

Figure, table and section ledger

Editorial contribution type: Theory & formal foundations (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF
21 physical pages
Before references / appendix
≈ 10.19 page-equivalents
Main figures / tables
0 / 0
Attached appendix figures / tables
0 / 0
Attached appendix span
≈ 7.89 page-equivalents
Before references / appendix: ≈ 48.52%References / intervening material: ≈ 13.9%Attached appendix: ≈ 37.57%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.
Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level headingStarts on PDF pageEstimated spanShare of entire PDF
1 INTRODUCTION1≈ 1.15 pp.5.48%
2 PRELIMINARIES2≈ 3.38 pp.16.1%
3 THE SIZE OF SMALLEST WITNESS VIA NON-EMPTINESS PROBLEM6≈ 2.7 pp.12.86%
4 SUCCINCTNESS ACROSS REPRESENTATIONS8≈ 1.43 pp.6.81%
5 APPLICATIONS10≈ 0.17 pp.0.81%
6 CONCLUDING REMARKS10≈ 0.71 pp.3.38%
ACKNOWLEDGMENTS11≈ 0.09 pp.0.43%
REFERENCES11≈ 2.91 pp.13.86%
A PROOFS14≈ 7.89 pp.37.57%
All detected figure and table locations

Main text

    No recognized caption labels.

    Attached appendix

      No recognized caption labels.

      Argument and organization

      1. Succinctness versus expressive capacity

        A distinction between recognizable languages and representation size motivates formal definitions and a worked counter.

      2. Verification complexity and representation gaps

        A tiling reduction and translation bounds establish results reused for LTL, automata and fixed-precision RNN comparisons.

      Problem, claim and contribution

      The opening resolves an apparent contradiction: a recognizer can express fewer languages yet describe some shared languages much more compactly. This cleanly separates the research question from raw expressive power.

      Pages 1–2, Section 1. · Main paper

      Argument and section order

      Definitions distinguish existential witness-family lower bounds from universal translation upper bounds. This prevents a worst-case separation from being read as an advantage on every language.

      Page 5, Definitions 2–3. · Main paper

      Figures, captions and visual explanation

      There are no empirical figures or tables. Instead, Example 2 uses a displayed four-bit counter and two annotated attention equations as the concrete explanatory object before the general tiling reduction.

      Page 6, Example 2 and Equations 7–8. · Main paper

      Tables, comparisons and evidence

      Theorem 15 breaks its witness construction into three numbered steps and then separately proves the LTL lower bound; Theorem 17 explicitly reuses the same witness. The page layout exposes the dependency rather than repeating the whole construction.

      Page 9, Theorems 15 and 17. · Main paper

      Appendix and reproducibility

      Proofs are named after their main-text statements; Appendix A.2 additionally gives a five-gadget plan, while A.4 separates numerator and denominator bounds. These local roadmaps make long symbolic arguments auditable.

      Appendix A.2, pages 14–17; A.4, pages 18–19. · Main paper

      Clarity and writing conventions

      The conclusion usefully limits the result to the specified hard-attention abstraction and treats succinctness of fixed-precision softmax models as future work. This narrower statement should govern reuse of the broad title.

      Page 10, Section 6; page 3 precision footnote. · Main paper

      What to reuse—and what not to infer

      Transferable lesson: For a theory paper, use one small worked construction, state quantifier direction explicitly, and organize each theorem around a visible chain of reductions and reused witnesses.

      Boundary: This is worst-case language-recognizer theory, not evidence of practical compression or trainability. UHAT, masking, tie-breaking and precision assumptions matter. The appendix's printed Equation 14e uses a conjunction between counter-increment and wrap conditions where the intended disjunction warrants checking; reading the proofs is not independent formal verification.

      Apply this lesson using the argument and evidence worksheet.

      Inspection record and sources

      Checked
      2026-09-19
      Reading status
      Full text + selected visuals inspected
      Main text
      Pages 1–10, Sections 1–6; acknowledgments and references on pages 11–13.
      Appendix / supplement
      Pages 14–21, Appendix A.1–A.7: completeness, tiling encoding, B-RASP translation, rational-bit bounds, LTL conversion, reverse translation and equivalence proofs.
      Visually inspected PDF pages
      6, 9