Writing study: To Infinity and Beyond: Tool-Use Unlocks Length Generalization in State Space Models

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Eran Malach, Omid Saremi, Sinead Williamson, Arwen Bradley, Aryo Lotfi, Emmanuel Abbe, Joshua Susskind, Etai Littwin · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Theory & formal foundations (primary) · Architecture & representation design.

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

27 physical pages

Before references / appendix

≈ 10.11 page-equivalents

Main figures / tables

3 / 1

Attached appendix figures / tables

7 / 1

Attached appendix span

≈ 12.73 page-equivalents

Before references / appendix: ≈ 37.44%References / intervening material: ≈ 15.41%Attached appendix: ≈ 47.15%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 2.78 pp.	10.3%
2 THEORY	4	≈ 2.84 pp.	10.52%
3 EXPERIMENTS	7	≈ 3.25 pp.	12.04%
4 CONCLUSION AND DISCUSSION	10	≈ 0.6 pp.	2.22%
REFERENCES	11	≈ 4.17 pp.	15.44%
A MORE DEFINITIONS	15	≈ 0.4 pp.	1.48%
B PROOFS	15	≈ 3.07 pp.	11.37%
C ARCHITECTURE AND TRAINING DETAILS	18	≈ 0.63 pp.	2.33%
D SYNTHETIC EXPERIMENTS DETAILS	19	≈ 7.28 pp.	26.96%
E CODE FIXING AGENT SETUP	26	≈ 0.64 pp.	2.37%
F TOOL USE CAPABILITIES OF PRETRAINED SSMS	27	≈ 0.7 pp.	2.59%

All detected figure and table locations

Main text

• Figure 1 · p. 2
• Figure 2 · p. 3
• Table 1 · p. 7
• Figure 3 · p. 10

Attached appendix

• Figure 4 · p. 20
• Table 2 · p. 22
• Figure 5 · p. 22
• Figure 6 · p. 23
• Figure 7 · p. 24
• Figure 8 · p. 25
• Figure 9 · p. 25
• Figure 10 · p. 26

Argument and organization

1. Long-form generation and GSSMs

   Defines the problem and model class before stating the limitation.

2. Impossibility and interactive construction

   Pairs a negative theorem with a precisely changed system boundary.

3. Synthetic and coding experiments

   Progresses from controlled algorithms to teacher-generated interactive traces.

Problem, claim and contribution

The paper turns an architectural limitation into a systems question: fixed internal memory is not fixed total memory when a model can repeatedly read and write an external store. That boundary change is the essential contribution, not merely adding a longer prompt.

Introduction, pp.1–3; discussion, p.10 · Main paper

Argument and section order

Definitions of output-support growth, finite-state models and three interaction protocols precede the theorems. The positive result explicitly uses constructed task-dependent traces and a string-matching learner; it is not a theorem about SGD-trained Mamba.

§2, pp.4–6; AppendixB, pp.15–18 · Main paper

Figures, captions and visual explanation

Figure1 aligns three trajectory formats with an extrapolation plot and marks the training limit at16 files. Figure6 uses consistent architecture colors while separating task difficulty, making successful arithmetic extrapolation and weaker Hanoi results visible together.

Figure1, p.2; Figure6, p.23 · Main paper

Tables, comparisons and evidence

The experiment specifies exact trajectory recovery rather than only final-answer correctness. Reported synthetic results select the best model after tuning, and Hanoi uses unequal seed counts, so the headline table is not a mean-over-seeds comparison.

§3.1 and Table1 footnote, p.7; AppendixC–D.4, pp.19–22 · Main paper

Appendix and reproducibility

The coding-agent prompt discloses the bug and prescribed repair strategy; successful shorter traces are filtered before student training. These details prevent the more realistic coding setting from being mistaken for unrestricted software engineering.

§3.3, p.9; AppendixE, pp.26–27 · Main paper

Clarity and writing conventions

The term computationally tractable is defined here by a halting Turing machine, without a polynomial-time requirement. Infinite generalization is a theoretical property under those definitions; experiments stop at specified sizes such as1,000 digits.

§2.4, p.6; §3.1, pp.7–8 · Main paper

What to reuse—and what not to infer

Transferable lesson: When a system overcomes a model limitation, state exactly which resource boundary changed and keep the theorem, training construction and empirical extrapolation ranges separate.

Boundary: The fixed-memory theorem assumes a finite state space and its particular output protocol. Tool storage and execution are not free, best-seed reporting is not average reliability, and the coding task is deliberately generated and narrowly specified.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

pp.1–10, §§1–4; references pp.11–15

Appendix / supplement

pp.15–27, appendices A–F covering formal protocols, proofs, task generators, ablations and coding-agent prompt

Visually inspected PDF pages

2, 23

• official ICLR 2026 proceedings PDF · 27 pages