Writing study: POET-X: Memory-efficient LLM Training by Scaling Orthogonal Transformation

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Zeju Qiu, Lixin Liu, Adrian Weller, Han Shi, Weiyang Liu · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Efficiency, inference & systems (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

15 physical pages

Before references / appendix

≈ 8.2 page-equivalents

Main figures / tables

6 / 10

Attached appendix figures / tables

1 / 5

Attached appendix span

≈ 4.9 page-equivalents

Before references / appendix: ≈ 54.67%References / intervening material: ≈ 12.67%Attached appendix: ≈ 32.67%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Source-bound correction: The actual appendix envelope starts at the p11 Appendix/Table of Contents cover, not A on p12. Table13 was anchored to a prose cross-reference on p12; restore its actual caption later on that page. Join wrapped section3.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 1.03 pp.	6.87%
2. Preliminaries of POET	2	≈ 0.44 pp.	2.93%
3. POET-X: Fast, Memory-efficient Training by Scaling Orthogonal Transformation	2	≈ 3.51 pp.	23.4%
4. Experiments and Results	6	≈ 2.59 pp.	17.27%
5. Related Work and Concluding Remarks	8	≈ 0.26 pp.	1.73%
Impact Statement	9	≈ 0.05 pp.	0.33%
References	9	≈ 1.91 pp.	12.73%
Appendix	11	≈ 0.98 pp.	6.53%
A. Experimental Details	12	≈ 2 pp.	13.33%
B. Additional Results	14	≈ 1.91 pp.	12.73%

All detected figure and table locations

Main text

• Figure 1 · p. 2
• Table 1 · p. 3
• Table 2 · p. 3
• Table 3 · p. 4
• Table 4 · p. 4
• Figure 2 · p. 4
• Table 5 · p. 5
• Figure 3 · p. 6
• Figure 4 · p. 6
• Figure 5 · p. 6
• Table 6 · p. 6
• Table 7 · p. 6
• Table 8 · p. 7
• Table 9 · p. 8
• Figure 6 · p. 8
• Table 10 · p. 8

Attached appendix

• Table 11 · p. 12
• Table 12 · p. 12
• Table 13 · p. 12
• Table 14 · p. 14
• Figure 7 · p. 14
• Table 15 · p. 15

Argument and organization

1. 1 Introduction; 2 Preliminaries

   Identifies the gap between sparse trainable parameters and actual memory use.

2. 3 POET-X

   Decomposes system improvements into algebraic reformulation, permutations, block computation, kernels, checkpointing and quantization.

3. 4 Experiments; 5 Concluding Remarks

   Moves from single-layer profiling to model training and multi-node scaling.

Problem, claim and contribution

The introduction defines practical memory overhead as the failure of an otherwise parameter-efficient formulation.

Introduction, p.1 · Main paper

Argument and section order

Each systems change is accompanied by its computational rationale, with fast and memory-saving variants explicitly separated.

Sections 3.1-3.5, pp.3-5 · Main paper

Figures, captions and visual explanation

The block construction diagram exposes tensor structure; a later page aligns memory tables with throughput curves that distinguish measured from ideal scaling.

Figure 2, p.4; Figure 6 and Tables 9-10, p.8 · Main paper

Tables, comparisons and evidence

The wall-clock comparison discloses DDP for the proposed method and FSDP for AdamW because of memory constraints, and the text retains AdamW’s stronger short-sequence single-GPU throughput.

Sections 4.2-4.3, pp.7-8 · Main paper

Appendix and reproducibility

Throughput measurement specifies warm-up and measured steps, excludes padding from token counts and discloses FFN dimension adjustments for block divisibility.

Appendix A, p.13; full throughput matrix, p.14 · Main paper

Clarity and writing conventions

The method states that the truncated Cayley-Neumann construction is approximately orthogonal, qualifying the spectrum-preserving motivation.

Section 3.4, p.4 · Main paper

What to reuse—and what not to infer

Transferable lesson: Connect each implementation change to a measured bottleneck, and disclose system configurations that explain both wins and losses.

Boundary: Microkernel speed-ups are not whole-training speed-ups; distributed strategy, block size, precision and sequence length materially affect the comparison.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-oriented scan of the main argument through physical p.9, with focused physical-page text inspection on 1, 3, 5, 7, 8, 9. Framing, section progression, mechanism/proof explanation, displays, comparison conditions and conclusions were inspected. This is not exhaustive technical reading, proof verification or reproduction.

Appendix / supplement

Appendix organization and main-text pointers were checked; focused text inspection of physical pages 13, 14. Remaining appendix proofs and details were not exhaustively read. No separately linked supplement inspected.

Visually inspected PDF pages

4, 8

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published proceedings PDF, PMLR volume 306, retrieved 2026-10-07 · 15 pages