Writing study: Revela: Dense Retriever Learning via Language Modeling

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Fengyu Cai, Tong Chen, Xinran Zhao, Sihao Chen, Hongming Zhang, Sherry Wu, Iryna Gurevych, Heinz Koeppl · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Learning & optimization algorithms (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

24 physical pages

Before references / appendix

≈ 10.11 page-equivalents

Main figures / tables

5 / 2

Attached appendix figures / tables

2 / 17

Attached appendix span

≈ 9.89 page-equivalents

Before references / appendix: ≈ 42.12%References / intervening material: ≈ 16.67%Attached appendix: ≈ 41.21%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 1.52 pp.	6.33%
2 RELATED WORKS	3	≈ 0.88 pp.	3.67%
3 R E V E L A : DENSE RETRIEVER LEARNING VIA LANGUAGE MODELING	4	≈ 1.58 pp.	6.58%
4 EXPERIMENTS	5	≈ 2.65 pp.	11.04%
5 ANALYSIS	8	≈ 1.47 pp.	6.13%
6 FUTURE DIRECTIONS	9	≈ 0.4 pp.	1.67%
7 CONCLUSION	10	≈ 0.22 pp.	0.92%
REPRODUCIBILITY STATEMENT	10	≈ 0.2 pp.	0.83%
ACKNOWLEDGMENTS	10	≈ 0.47 pp.	1.96%
REFERENCES	11	≈ 4 pp.	16.67%
A ILLUSTRATION	15	≈ 0.68 pp.	2.83%
B EXPERIMENTS	15	≈ 3.89 pp.	16.21%
C ANALYSIS	19	≈ 4.6 pp.	19.17%
D USE OF LLMS	24	≈ 0.73 pp.	3.04%

All detected figure and table locations

Main text

• Figure 1 · p. 2
• Figure 2 · p. 4
• Table 1 · p. 7
• Figure 3 · p. 7
• Table 2 · p. 8
• Figure 4 · p. 9
• Figure 5 · p. 9

Attached appendix

• Figure 6 · p. 15
• Table 3 · p. 16
• Table 4 · p. 16
• Figure 7 · p. 17
• Table 5 · p. 18
• Table 6 · p. 18
• Table 7 · p. 19
• Table 8 · p. 19
• Table 9 · p. 20
• Table 10 · p. 20
• Table 11 · p. 20
• Table 12 · p. 21
• Table 13 · p. 21
• Table 14 · p. 21
• Table 15 · p. 22
• Table 16 · p. 22
• Table 17 · p. 23
• Table 18 · p. 23
• Table 19 · p. 23

Argument and organization

1. Motivation and in-batch language modeling

   Turns the analogy between token dependence and document dependence into a differentiable retrieval objective.

2. Benchmarking followed by controlled analysis

   Separates model comparisons from same-backbone, same-data controls.

Problem, claim and contribution

The introduction makes the missing supervision problem concrete before proposing next-token prediction as the training signal, rather than introducing another retrieval loss without motivation.

Sections 1 and 3.1, pages 1–4. · Main paper

Figures, captions and visual explanation

Figure 2 puts document embeddings, a similarity matrix, the transformer block, and the two-stream attention mask in one compact diagram; the green/pink legend is essential to interpreting information access.

Figure 2, page 4. · Main paper

Tables, comparisons and evidence

Table 2 holds the Llama-1B backbone and training domain fixed, a more diagnostic comparison than the heterogeneous model leaderboard.

Section 5 and Table 2, page 8. · Main paper

The reported data-efficiency comparison counts fine-tuning data, not all data used to pretrain the language-model backbones; it should not be summarized as training from scratch with 1,000 times less data.

Section 4.2, pages 7–8; Appendix B.4, page 18. · Main paper

Appendix and reproducibility

Corpus examples, exact checkpoint links and licenses, V-normalization, and per-task tables make the training setup inspectable; domain-specific design changes are not hidden behind a single default.

Appendix B.2–B.6, pages 16–19. · Main paper

Clarity and writing conventions

Bold result-led paragraph headings guide the analysis, but the larger-LM headline needs the per-task qualifications: BEIR averages do not consistently improve with LM size.

Section 5, pages 8–9; Appendix C.3, Table 13. · Main paper

What to reuse—and what not to infer

Transferable lesson: Pair an intuitive reformulation with an explicit information-flow diagram, then isolate its contribution using matched backbone and data controls.

Boundary: Empirical size trends are not a proven scaling law, and the document-count efficiency claim does not include backbone pretraining. This joint training method is not simply frozen-teacher distillation.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

Pages 1–10, Sections 1–6; references on pages 11–14.

Appendix / supplement

Pages 15–24, Appendices A–D, including all benchmark tables and implementation variants.

Visually inspected PDF pages

4, 8

• Official ICLR 2026 proceedings conference PDF; PDF creation metadata: Sat May 9 08:35:06 2026 EDT · 24 pages