Writing study: ECHO: Elastic Speculative Decoding with Sparse Gating for High-Concurrency Scenarios

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Xinyi. Hu, Yuhao Shen, Zhang Baolin, Hengxin Zhang, Jun Dai, Shuang Ge, Chen Lei, Yue Li, Mingcheng Wan · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Efficiency, inference & systems (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

21 physical pages

Before references / appendix

≈ 8.92 page-equivalents

Main figures / tables

5 / 1

Attached appendix figures / tables

4 / 7

Attached appendix span

≈ 9.91 page-equivalents

Before references / appendix: ≈ 42.48%References / intervening material: ≈ 10.33%Attached appendix: ≈ 47.19%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 1.56 pp.	7.43%
2. Problem Formulation	2	≈ 0.5 pp.	2.38%
3. Method	3	≈ 2.4 pp.	11.43%
4. Theoretical Motivation	5	≈ 0.58 pp.	2.76%
5. Experiments	6	≈ 2.84 pp.	13.52%
6. Related Works	9	≈ 0.39 pp.	1.86%
7. Conclusion	9	≈ 0.1 pp.	0.48%
8. Limitations	9	≈ 0.14 pp.	0.67%
Impact Statement	9	≈ 0.05 pp.	0.24%
References	9	≈ 2.16 pp.	10.29%
A. Theoretical Analysis	12	≈ 2 pp.	9.52%
B. Extended Related Works	14	≈ 1 pp.	4.76%
C. Evaluation Details	15	≈ 2 pp.	9.52%
D. Additional Experimental Results	17	≈ 2 pp.	9.52%
E. Layer-wise Confidence Density Visualization for Each Model	19	≈ 2.91 pp.	13.86%

All detected figure and table locations

Main text

• Figure 1 · p. 1
• Figure 2 · p. 2
• Figure 3 · p. 4
• Table 1 · p. 7
• Figure 4 · p. 7
• Figure 5 · p. 8

Attached appendix

• Table 2 · p. 16
• Table 3 · p. 17
• Table 4 · p. 17
• Table 5 · p. 17
• Table 6 · p. 18
• Table 7 · p. 18
• Table 8 · p. 18
• Figure 6 · p. 20
• Figure 7 · p. 20
• Figure 8 · p. 21
• Figure 9 · p. 21

Argument and organization

1. Problem and contribution

   Explains why maximizing accepted draft length alone fails once verification becomes compute-bound at high concurrency.

2. Argument sequence

   A global budget constraint leads to sparse gating and an explicit depth-before-width priority, then evaluation separates low-load from high-load regimes.

3. Evidence and verification

   Dense-gating and fixed-threshold variants test different decisions. Baselines are adapted to the same EAGLE-3 backbone, and hardware/framework choices are explicitly workload-dependent.

Problem, claim and contribution

Explains why maximizing accepted draft length alone fails once verification becomes compute-bound at high concurrency.

Introduction and Problem Formulation, pp. 2-3 · Main paper

Argument and section order

A global budget constraint leads to sparse gating and an explicit depth-before-width priority, then evaluation separates low-load from high-load regimes.

Sections 2-5, pp. 2-8 · Main paper

Figures, captions and visual explanation

The formulation page names the shared batch budget; the main grid aligns benchmark rows and model columns, using a shared legend and visibly broken low/high-batch panels.

Equations 1-4, p. 3; Figure 5, p. 8; both PNGs inspected · Main paper

Tables, comparisons and evidence

Dense-gating and fixed-threshold variants test different decisions. Baselines are adapted to the same EAGLE-3 backbone, and hardware/framework choices are explicitly workload-dependent.

Section 5.3, p. 8; Appendix C.2-C.4, p. 15 · Main paper

Appendix and reproducibility

Deployment checks extend calibration, context, hardware and latency behavior; the limitations restrict overhead claims to evaluated drafters and leave online calibration/ultra-long contexts open.

Appendix D, pp. 17-18; Sections 5.4 and 8, p. 9 · Main paper

Clarity and writing conventions

The paper labels its formal results motivation rather than a complete scheduling optimality theorem, and distinguishes conceptual composability from direct scheduler comparison.

Section 4, pp. 5-6; related work, p. 9 · Main paper

What to reuse—and what not to infer

Transferable lesson: Organize a serving paper around the actual scarce resource and workload regimes, with controlled baseline implementations and explicit deployment limits.

Boundary: Training-free gating still uses a pretrained drafter and startup calibration; throughput on the evaluated stack does not guarantee per-request fairness or production behavior on other drafters.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-focused distributed manuscript scan, not exhaustive technical/proof validation. Sections 1-8, pp. 1-9: serving bottleneck, fixed verification budget, sparse gates, elastic scheduler, formal motivation, load-dependent tests and limitations.

Appendix / supplement

Writing-focused appendix scan; code, data and external assets not executed. Appendices A-E, pp. 12-21: formal motivation, related work, baseline adaptations and hardware setup, calibration/deployment checks and depth confidence plots.

Visually inspected PDF pages

3, 8

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published ICML 2026 proceedings PDF, PMLR volume 306; retrieved 2026-10-07. Public OpenReview decision notes were not independently reverified. · 21 pages