Writing study: Overcoming Joint Intractability with Lossless Hierarchical Speculative Decoding

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Yuxuan Zhou, Fei Huang, Heng Li, Fengyi Wu, Tianyu Wang, jianwei zhang, Junyang Lin, Zhi-Qi Cheng · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Inference, algorithms & systems efficiency (primary) · Theory & formal foundations.

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

34 physical pages

Before references / appendix

≈ 10.21 page-equivalents

Main figures / tables

2 / 4

Attached appendix figures / tables

1 / 4

Attached appendix span

≈ 21.4 page-equivalents

Before references / appendix: ≈ 30.03%References / intervening material: ≈ 7.03%Attached appendix: ≈ 62.94%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 1.12 pp.	3.29%
2 RELATED WORK	2	≈ 0.52 pp.	1.53%
3 REVISITING TOKENWISE SPECULATIVE DECODING	3	≈ 0.42 pp.	1.24%
4 THEORETICAL FOUNDATIONS OF HIERARCHICAL SPECULATIVE DECODING	3	≈ 2.05 pp.	6.03%
5 HIERARCHICAL SPECULATIVE DECODING	5	≈ 3.5 pp.	10.29%
6 EXPERIMENTS	9	≈ 1.67 pp.	4.91%
7 CONCLUSION	10	≈ 0.28 pp.	0.82%
ACKNOWLEDGMENTS	11	≈ 0.11 pp.	0.32%
REFERENCES	11	≈ 2.39 pp.	7.03%
APPENDIX	13	≈ 0.04 pp.	0.12%
A THEORETICAL FOUNDATION	13	≈ 2 pp.	5.88%
B LOSSLESS OF NAIVE HIERARCHICAL SPECULATIVE DECODING	15	≈ 3.47 pp.	10.21%
C LOSSLESS OF HIERARCHICAL SPECULATIVE DECODING	19	≈ 4.38 pp.	12.88%
D EXPECTED NUMBER OF ACCEPTED TOKENS	23	≈ 4.96 pp.	14.59%
E EXTENDED EXPERIMENTS	28	≈ 0.22 pp.	0.65%
F PYTHON IMPLEMENTATION	28	≈ 3.43 pp.	10.09%
G INTEGRATION WITH RECURSIVE REJECT SAMPLING IN THE MULTI-DRAFT SETUP	32	≈ 1 pp.	2.94%
H COMPUTATION EFFICIENCY	33	≈ 1.89 pp.	5.56%

All detected figure and table locations

Main text

• Figure 1 · p. 2
• Figure 2 · p. 8
• Table 1 · p. 9
• Table 2 · p. 10
• Table 3 · p. 10
• Table 4 · p. 10

Attached appendix

• Figure A.1 · p. 23
• Table A.1 · p. 28
• Table A.2 · p. 28
• Table A.3 · p. 28
• Table A.4 · p. 34

Argument and organization

1. Tokenwise recovery as a baseline

   Derives the familiar accept/resample decomposition before extending it to branches.

2. Naive hierarchy then capped hierarchy

   Separates correctness intuition from the computationally feasible algorithm.

3. Proofs, examples and implementation

   Connects symbolic conditions to a concrete token sequence and hardware measurements.

Problem, claim and contribution

Joint intractability is named as the specific bottleneck, which distinguishes a verifier contribution from another draft-model improvement.

Section 1, p. 1 · Main paper

Argument and section order

Two adjacent algorithms expose exactly what changes between the costly naive construction and the capped method.

Algorithms 1–2, p. 5 · Main paper

Figures, captions and visual explanation

A backward-scan schematic uses distinct arrows for rejection and corrective sampling; its caption describes the accepted prefix and the single resampling location.

Figure 1, p. 2 · Main paper

Tables, comparisons and evidence

Tables keep hardware-independent tokens per target step separate from tokens per second, avoiding an automatic equivalence between acceptance and latency.

Tables 1–4, pp. 9–10 · Main paper

Appendix and reproducibility

The supplement includes a numerical recovery example, full proof route, Python listings, multi-draft pseudocode and component timing, making distinct verification levels accessible.

Appendices A–H, pp. 13–34 · Main paper

Clarity and writing conventions

The EAGLE integration paragraph qualifies the headline speedup: it keeps first-token acceptance and attributes gains largely to implementation, rather than stronger acceptance theory.

Section 6.2, p. 10 · Main paper

What to reuse—and what not to infer

Transferable lesson: Present a theoretically simple but impractical algorithm first, then show the exact modification that removes its overhead; report acceptance and measured latency separately.

Boundary: Do not attach the standalone lossless guarantee to every integration. The EAGLE experiment keeps a non-lossless first-token convention; the printed code is illustrative and includes unresolved variables, so it is not a verified executable artifact.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

Pages 1–10, Sections 1–7, including all definitions, algorithms and worked example

Appendix / supplement

Pages 13–34, Appendices A–H: proofs, expected acceptance, extended experiments, code listings, multi-draft algorithm and runtime accounting

Visually inspected PDF pages

2, 10

• ICLR 2026 final proceedings · 34 pages