Writing study: Non-Convex Federated Optimization under Cost-Aware Client Selection

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Xiaowen Jiang, Anton Rodomanov, Sebastian Stich · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Theory & formal foundations (primary) · Learning & optimization algorithms · Inference, algorithms & systems efficiency.

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

43 physical pages

Before references / appendix

≈ 10.11 page-equivalents

Main figures / tables

2 / 1

Attached appendix figures / tables

10 / 4

Attached appendix span

≈ 29.9 page-equivalents

Before references / appendix: ≈ 23.51%References / intervening material: ≈ 6.95%Attached appendix: ≈ 69.53%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 1.19 pp.	2.77%
2 PROBLEM FORMULATION	2	≈ 1.32 pp.	3.07%
3 INEXACT COMPOSITE GRADIENT METHOD	4	≈ 2.2 pp.	5.12%
4 BASIC APPLICATION EXAMPLES: SAGA + SVRG	6	≈ 1.51 pp.	3.51%
5 RECURSIVE GRADIENT ESTIMATOR + EXAMPLES (SAGA AND SVRG)	7	≈ 1.45 pp.	3.37%
6 COMMUNICATION AND LOCAL COMPLEXITY OF I-CGM-RG	9	≈ 0.32 pp.	0.74%
7 NUMERICAL EXPERIMENTS	9	≈ 1.17 pp.	2.72%
8 CONCLUSION	10	≈ 0.36 pp.	0.84%
REFERENCES	11	≈ 3 pp.	6.98%
Appendix	14	≈ 1 pp.	2.33%
A ETHICS STATEMENT	15	≈ 0.08 pp.	0.19%
B LLM USAGE	15	≈ 0.92 pp.	2.14%
C FEDERATED OPTIMIZATION ALGORITHMS AND THEIR COMPLEXITY	16	≈ 1.74 pp.	4.05%
D RELATED WORK	17	≈ 3.96 pp.	9.21%
E TECHNICAL PRELIMINARIES	21	≈ 1.3 pp.	3.02%
F PROOFS FOR I-CGM	23	≈ 9.61 pp.	22.35%
G I-CGM-RG-SAGA WITH INEXACT INITIALIZATION	32	≈ 4.39 pp.	10.21%
H SOLVING THE SUBPROBLEMS WITH LOCAL STOCHASTIC CGM	37	≈ 1.53 pp.	3.56%
I DISCUSSION ON THE SAG ESTIMATOR	38	≈ 0.84 pp.	1.95%
J ADDITIONAL DETAILS AND EXPERIMENTS	39	≈ 4.51 pp.	10.49%

All detected figure and table locations

Main text

• Table 1 · p. 3
• Figure 1 · p. 10
• Figure 2 · p. 10

Attached appendix

• Figure D.1 · p. 18
• Figure J.1 · p. 39
• Figure J.2 · p. 39
• Figure J.3 · p. 40
• Figure J.4 · p. 40
• Figure J.5 · p. 41
• Figure J.6 · p. 41
• Table J.1 · p. 41
• Table J.2 · p. 41
• Figure J.7 · p. 42
• Figure J.8 · p. 42
• Figure J.9 · p. 42
• Table J.3 · p. 43
• Table J.4 · p. 43

Argument and organization

1. Cost-aware model and inexact composite-gradient framework

   Define the resource being optimized before the estimator analysis

2. SAGA/SVRG variance bounds and recursive correction

   Build the main convergence result from modular lemmas

3. Experiments and extensive appendix

   Separate abstract costs, practical simulations, proofs and extensions

Problem, claim and contribution

The motivating distinction is costly arbitrary client selection versus cheaper random selection and a reliable delegate, not simply fewer communication rounds. This makes the resource model part of the contribution.

Sections 1–2; Appendix C, pages 16–17 · Main paper

Argument and section order

Theorem 3.1 isolates acceptable subproblem and estimator errors before SAGA variance and recursive-gradient lemmas instantiate them. This modular chain helps readers distinguish the framework from its two realizations.

Sections 3–6; Appendix F, pages 23–32 · Main paper

Figures, captions and visual explanation

Figure 1 pairs communication and local-computation axes; Figure J.8 adds validation accuracy and empirical smoothness/similarity traces. These complementary panels expose the trade-off rather than hiding local work behind round counts.

Page 10, Figure 1; page 42, Figure J.8 · Main paper

Tables, comparisons and evidence

The CIFAR-10 table reports best validation accuracy within 100 outer iterations: SAGA 76.1, SVRG 77.0 and FedAvg 74.3. Thus the theoretical favorite is not uniformly the empirical winner; neural experiments also introduce tuned control-variate damping.

Appendix J.3–J.4, Tables J.3–J.4, pages 41–43 · Main paper

Appendix and reproducibility

Appendix G makes the price of avoiding initial full synchronizations explicit through additional error terms; J separately varies selection-cost ratios. The long appendix supplies substantive alternatives, not merely omitted algebra.

Appendix G, pages 32–36; Figure J.5, page 41 · Main paper

Clarity and writing conventions

Important modeling details are deferred to an appendix the reader is encouraged to consult early. The final limitations correctly disclose the reliable-delegate assumption and unconstrained transmitted message size; those boundaries deserve main-text prominence.

Section 2; Appendix C; page 43, Limitations and Future Extensions · Main paper

What to reuse—and what not to infer

Transferable lesson: For optimization writing, align assumptions, complexity tables and experimental axes, and explicitly price initialization and local work rather than reporting rounds alone.

Boundary: Weighted oracle/selection cost is not measured network bytes, latency or privacy. Reading the proofs is not independent proof certification; simulated neural-network tuning does not establish every theoretical assumption.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

Pages 1–10, Sections 1–8; references 11–13

Appendix / supplement

Pages 14–43, contents and Appendices A–J, including all proofs, initialization variants, stochastic-local solver and experimental limitations

Visually inspected PDF pages

10, 42

• official ICLR 2026 proceedings PDF · 43 pages