Writing study: Online Learning and Equilibrium Computation with Ranking Feedback

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Mingyang Liu, Yongshan Chen, Zhiyuan Fan, Gabriele Farina, Asuman Ozdaglar, Kaiqing Zhang · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Theory & formal foundations (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

46 physical pages

Before references / appendix

≈ 10.59 page-equivalents

Main figures / tables

2 / 1

Attached appendix figures / tables

9 / 0

Attached appendix span

≈ 31.9 page-equivalents

Before references / appendix: ≈ 23.02%References / intervening material: ≈ 7.63%Attached appendix: ≈ 69.35%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 1.24 pp.	2.7%
2 PRELIMINARIES	2	≈ 0.97 pp.	2.11%
3 ONLINE LEARNING WITH RANKING FEEDBACK	3	≈ 1.58 pp.	3.43%
4 HARDNESS RESULTS	5	≈ 1.06 pp.	2.3%
5 ONLINE LEARNING WITH InstUtil Rank FEEDBACK	6	≈ 1.18 pp.	2.57%
6 ONLINE LEARNING WITH AvgUtil Rank FEEDBACK	7	≈ 1.68 pp.	3.65%
7 EQUILIBRIUM COMPUTATION WITH RANKING FEEDBACK	9	≈ 1.17 pp.	2.54%
8 EXPERIMENTS	10	≈ 0.72 pp.	1.57%
9 ETHICS STATEMENT	11	≈ 0.19 pp.	0.41%
10 REPRODUCIBILITY STATEMENT	11	≈ 0.08 pp.	0.17%
11 USE OF LARGE LANGUAGE MODELS	11	≈ 0.08 pp.	0.17%
REFERENCES	11	≈ 3.52 pp.	7.65%
A RELATED WORK	15	≈ 1.05 pp.	2.28%
B ADDITIONAL NOTATION AND PRELIMINARIES	16	≈ 1.24 pp.	2.7%
C ADDITIONAL EXPERIMENTS	17	≈ 1.05 pp.	2.28%
D PROOF OF SECTION 4	18	≈ 5.4 pp.	11.74%
E PROOF OF THEOREM 5.1	23	≈ 7.03 pp.	15.28%
F ALGORITHMS AND DIAGRAMS	30	≈ 1.79 pp.	3.89%
G PROOF OF THEOREM 5.2 (FULL-INFORMATION)	32	≈ 1.62 pp.	3.52%
H PROOF OF THEOREM 5.2 (BANDIT)	34	≈ 2.57 pp.	5.59%
I PROOF OF THEOREM 6.2	36	≈ 1.25 pp.	2.72%
J PROOF OF THEOREM 6.3	38	≈ 5.38 pp.	11.7%
K PROOF OF SECTION 7	43	≈ 1.96 pp.	4.26%
L PROPERTIES OF FOLLOW-THE-REGULARIZED-LEADER (FTRL)	45	≈ 1.27 pp.	2.76%
M CONCLUSION AND LIMITATIONS	46	≈ 0.29 pp.	0.63%

All detected figure and table locations

Main text

• Figure 1 · p. 2
• Table 1 · p. 3
• Figure 2 · p. 10

Attached appendix

• Figure 3 · p. 18
• Figure 4 · p. 18
• Figure 5 · p. 19
• Figure 6 · p. 19
• Figure 7 · p. 19
• Figure 8 · p. 20
• Figure 9 · p. 20
• Figure 10 · p. 31
• Figure 11 · p. 32

Argument and organization

1. Four feedback regimes and hardness

   Defines the comparison grid before stating the attainable guarantees.

2. Utility estimation → no-regret wrappers → equilibria

   Uses one estimation module to connect learning and game-theoretic consequences.

3. Routing illustration and proof appendices

   Separates an application demonstration from the theoretical core.

Problem, claim and contribution

The motivation asks what information numerical rewards provide that ordinal rankings lose. It carefully distinguishes instantaneous rankings from rankings of accumulated experience.

Sections 1–3, pp. 1–5 · Main paper

Argument and section order

Table 1 is a useful theorem map: full-information/bandit columns cross instantaneous/average rows, while upper and lower bounds occupy separate blocks with their temperature and variation assumptions.

Table 1, p. 3 · Main paper

Figures, captions and visual explanation

Appendix diagrams align the full-information and bandit pipelines vertically; the added uniform-exploration branch is immediately visible and the matching pseudocode appears on the same page.

Figure 10 and Algorithm 2, p. 31 · Main paper

Tables, comparisons and evidence

The LLM routing illustration uses reward-model scores to simulate feedback, not live human preferences. Synthetic experiments disclose ten seeds, parameter grids and selection of the best final-horizon configuration.

Section 8, p. 10; Appendix C, pp. 17–20 · Main paper

Appendix and reproducibility

Proof sections repeat the exact theorem being established and use labeled regret-decomposition terms, making the estimation-to-regret argument navigable despite its length.

Appendices D–L, pp. 18–46 · Main paper

Clarity and writing conventions

The paper explicitly discloses language-model assistance for prose, illustration and code, and closes with the unresolved average-ranking bandit gap rather than implying a complete characterization.

Section 11, p. 11; Appendix M, p. 46 · Main paper

What to reuse—and what not to infer

Transferable lesson: For a theory paper with several regimes, provide an assumption-by-result matrix and reusable algorithm diagrams before asking readers to navigate proofs.

Boundary: Do not present simulated reward-model rankings as a user study or a coarse correlated equilibrium as stable matching. The Appendix C Figure 3 caption calls its plotted online-regret curves exploitability, so metric labels require care.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

Pages 1–11, Sections 1–11, including all model definitions, theorem statements and routing experiment.

Appendix / supplement

Pages 15–46, Appendices A–M: related work, game definitions, experiments, algorithms, complete proofs and limitations.

Visually inspected PDF pages

3, 10, 31

• ICLR 2026 official proceedings PDF, retrieved 2026-09-19 · 46 pages