Writing study: Temporal superposition and feature geometry of RNNs under memory demands

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Pratyaksh Sharma, Alexandra M Proca, Lucas Prieto, Pedro Mediano · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Theory & formal foundations (primary) · Empirical analysis, diagnosis & auditing.

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

42 physical pages

Before references / appendix

≈ 10.3 page-equivalents

Main figures / tables

5 / 0

Attached appendix figures / tables

12 / 1

Attached appendix span

≈ 25.9 page-equivalents

Before references / appendix: ≈ 24.52%References / intervening material: ≈ 13.81%Attached appendix: ≈ 61.67%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 1.07 pp.	2.55%
2 KEY INTUITIONS: SPATIAL & TEMPORAL SUPERPOSITION	2	≈ 1.01 pp.	2.4%
3 MATHEMATICAL SETUP	3	≈ 1.61 pp.	3.83%
4 RESULTS	5	≈ 3.56 pp.	8.48%
5 RELATED WORK	8	≈ 1.66 pp.	3.95%
6 DISCUSSION	10	≈ 0.59 pp.	1.4%
ACKNOWLEDGMENTS	11	≈ 0.07 pp.	0.17%
REPRODUCIBILITY STATEMENT	11	≈ 0.13 pp.	0.31%
REFERENCES	11	≈ 5.8 pp.	13.81%
Appendix	17	≈ 1 pp.	2.38%
A ASSUMPTIONS ABOUT DATA	18	≈ 0.29 pp.	0.69%
B DECOMPOSING THE k-DELAY LOSS	18	≈ 2.16 pp.	5.14%
C GEOMETRIC INTERPRETATION OF LEARNING DYNAMICS	20	≈ 1.05 pp.	2.5%
D ANALYZING THE SPECTRAL RADIUS OF Wh	21	≈ 3.84 pp.	9.14%
E STATE SPACE MODEL (LINEAR RECURRENCE, NONLINEAR READOUT)	25	≈ 3.26 pp.	7.76%
F NONLINEAR RNN (NONLINEAR RECURRENCE, NONLINEAR READOUT)	28	≈ 2.03 pp.	4.83%
G HIGHER DIMENSIONAL RESULTS	30	≈ 4.46 pp.	10.62%
H TASKS WITH RANDOM DELAY	35	≈ 3.5 pp.	8.33%
I EXPERIMENTAL DETAILS	38	≈ 4.31 pp.	10.26%

All detected figure and table locations

Main text

• Figure 1 · p. 3
• Figure 2 · p. 6
• Figure 3 · p. 7
• Figure 4 · p. 7
• Figure 5 · p. 9

Attached appendix

• Figure 6 · p. 21
• Figure 7 · p. 23
• Figure 8 · p. 24
• Figure 9 · p. 25
• Figure 10 · p. 28
• Figure 11 · p. 30
• Figure 12 · p. 33
• Figure 13 · p. 34
• Figure 14 · p. 36
• Figure 15 · p. 37
• Figure 16 · p. 38
• Table 1 · p. 39
• Figure 17 · p. 40

Argument and organization

1. Intuition before formalization

   Separates spatial and temporal superposition with concrete feature diagrams.

2. Controlled model ladder

   Moves from linear recurrence to nonlinear readout and then nonlinear recurrence.

3. Limits and extensions

   Tests higher dimensions, interacting feature pressures and random delays after establishing the toy mechanism.

Problem, claim and contribution

The paper isolates memory duration as a capacity constraint, distinguishing when a feature occurred from what it represents before introducing algebra.

Section 2 and Figure 1, pages 2–3 · Main paper

Argument and section order

The sequence linear RNN → nonlinear readout → nonlinear recurrence changes one source of expressivity at a time, letting the reader attribute different geometries to a specific modeling choice.

Sections 4.1–4.4, pages 5–8 · Main paper

Figures, captions and visual explanation

Figure 1 combines shape-coded feature identity with color-coded age and separate interference sketches. Its caption defines each visual encoding and distinguishes projection from composition.

Figure 1, page 3 · Main paper

Tables, comparisons and evidence

Figure 2 aligns empirical loss, decomposed theoretical terms, readout projections and final geometry, creating a direct explanatory bridge between optimization behavior and the claimed mechanism.

Figure 2, page 6 · Main paper

Appendix and reproducibility

A detailed contents page and per-figure protocol explain model selection, weight tying, sparsity and optimization. The frequency-of-geometry table is especially useful because many main plots select low-loss runs.

Appendix I, pages 38–42; Table 1, page 39 · Main paper

Clarity and writing conventions

The discussion clearly acknowledges temporal independence, high sparsity and the fixed-delay toy task. Random-delay results are labeled preliminary instead of being presented as the same established mechanism.

Section 6 page 10; Appendix H pages 35–38 · Main paper

What to reuse—and what not to infer

Transferable lesson: A theory-driven paper can be taught through a controlled ladder of models and a shared visual vocabulary, with each theorem term linked to a measurable geometric behavior.

Boundary: Selected best-performing tiny models do not establish prevalence in large sequence models. Preserve sparsity and independence assumptions, and distinguish numerical evidence for optimal geometry from general proofs.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

Pages 1–10, Sections 1–6

Appendix / supplement

Pages 17–42, contents and Appendices A–I including derivations, additional geometries, random-delay task and per-figure protocols

Visually inspected PDF pages

3, 6

• ICLR 2026 final proceedings · 42 pages