Writing study: FlashSinkhorn: IO-Aware Entropic Optimal Transport on GPU

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Felix X.-F. Ye, Xingjie Li, An Yu, Ming-Ching Chang, Linsong Chu, Davis Wertheimer · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Efficiency, inference & systems (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

36 physical pages

Before references / appendix

≈ 9.16 page-equivalents

Main figures / tables

5 / 3

Attached appendix figures / tables

3 / 21

Attached appendix span

≈ 23.91 page-equivalents

Before references / appendix: ≈ 25.44%References / intervening material: ≈ 8.14%Attached appendix: ≈ 66.42%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 0.55 pp.	1.53%
2. Background	2	≈ 1.11 pp.	3.08%
3. FlashSinkhorn: Algorithm and Analysis	3	≈ 2.95 pp.	8.19%
4. Experiments	6	≈ 2.57 pp.	7.14%
5. Related Work	8	≈ 0.97 pp.	2.69%
6. Conclusion	9	≈ 0.29 pp.	0.81%
Acknowledgements	10	≈ 0.07 pp.	0.19%
Impact Statement	10	≈ 0.05 pp.	0.14%
References	10	≈ 2.92 pp.	8.11%
A. Notation	13	≈ 1 pp.	2.78%
B. Derivation of Sinkhorn algorithm	14	≈ 1.32 pp.	3.67%
C. Differentiation of EOT with respect to data	15	≈ 2.06 pp.	5.72%
D. Proof in Section 3.1	17	≈ 2.24 pp.	6.22%
E. Proof in Section 3.2	19	≈ 1.38 pp.	3.83%
F. Streaming Hessian-Vector Products Derivation and Proof of Theorem 5	21	≈ 2.27 pp.	6.31%
G. Implementation Details	23	≈ 1.27 pp.	3.53%
H. Full Experiment Results	24	≈ 12.37 pp.	34.36%

All detected figure and table locations

Main text

• Figure 1 · p. 4
• Figure 2 · p. 4
• Table 1 · p. 5
• Table 2 · p. 6
• Figure 3 · p. 7
• Table 3 · p. 7
• Figure 4 · p. 9
• Figure 5 · p. 9

Attached appendix

• Table 4 · p. 13
• Table 5 · p. 27
• Table 6 · p. 27
• Table 7 · p. 28
• Table 8 · p. 28
• Table 9 · p. 29
• Table 10 · p. 29
• Table 11 · p. 30
• Table 12 · p. 30
• Table 13 · p. 31
• Table 14 · p. 31
• Figure 6 · p. 32
• Table 15 · p. 32
• Table 16 · p. 33
• Table 17 · p. 33
• Table 18 · p. 34
• Table 19 · p. 34
• Table 20 · p. 35
• Table 21 · p. 35
• Table 22 · p. 36
• Table 23 · p. 36
• Table 24 · p. 36
• Figure 7 · p. 36
• Figure 8 · p. 36

Argument and organization

1. 1 Introduction; 2 Background

   Identifies memory traffic and establishes stabilized OT and differentiation notation.

2. 3 Algorithm and Analysis

   Builds streaming updates, transport application and Hessian-vector products from a shared attention correspondence.

3. 4 Experiments; 5 Related Work; 6 Conclusion

   Separates kernel benchmarks, numerical accuracy, downstream tasks and cost-class limitations.

Problem, claim and contribution

The bottleneck is GPU memory traffic for squared-Euclidean entropic OT, not a claim to remove the quadratic interaction arithmetic or support every transport cost.

Abstract, p.1; Conclusion, p.9 · Main paper

Argument and section order

Exact update identities precede the two-level memory model, and transport application distinguishes arbitrary potentials from converged optimal coupling.

Section 3.1, p.3; Section 3.2, p.5 · Main paper

Figures, captions and visual explanation

The kernel diagrams expose on-chip tiling and streaming, while the benchmark grid separates forward, backward, HVP, size, dimension and memory axes with a shared method legend.

Figures 1-2, p.4; Figure 3, p.7 · Main paper

Tables, comparisons and evidence

Comparisons align alternating versus symmetric schedules, and low-regularization results distinguish fixed-iteration throughput from total iterations needed for convergence.

Section 3.1, p.3; Table 3 and low-epsilon discussion, p.7 · Main paper

Appendix and reproducibility

The notation table separates induced and optimal couplings, while full experimental setup records numerical and timing choices needed to interpret kernel gains.

Appendix A, p.13; Appendix H.1, p.24 · Main paper

Clarity and writing conventions

Maximum forward-plus-backward speedup is not a uniform pipeline gain; the downstream saddle-escape panel separately shows the much smaller total wall-time change.

Table 3, p.7; Figure 5, p.9 · Main paper

What to reuse—and what not to infer

Transferable lesson: Carry exact algebra, memory complexity, finite precision and convergence budget through separate reporting layers instead of collapsing them into one speedup.

Boundary: Linear resident memory is not linear arithmetic; streaming identities can be exact while finite iterations and approximate CG still limit final numerical accuracy, and the largest gain is baseline/configuration-specific.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-oriented scan of the main argument through physical p.9, with focused physical-page text inspection on 1, 3, 5, 6, 7, 8, 9. Framing, section progression, mechanism/proof explanation, displays, comparison conditions and conclusions were inspected. This is not exhaustive technical reading, proof verification or reproduction.

Appendix / supplement

Appendix organization and main-text pointers were checked; focused text inspection of physical pages 13, 20, 24. Remaining appendix proofs and details were not exhaustively read. No separately linked supplement inspected.

Visually inspected PDF pages

4, 7

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published proceedings PDF, PMLR volume 306, retrieved 2026-10-07 · 36 pages