Writing study: DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Rulin Shao, Akari Asai, Shannon Zejiang Shen, Hamish Ivison, Varsha Kishore, Jingming Zhuo, Xinran Zhao, Molly Park, Samuel G. Finlayson, David Sontag, Tyler Murray, Sewon Min, Pradeep Dasigi, Luca Soldaini, Faeze Brahman, Wen-Tau Yih, Tongshuang Wu, Luke Zettlemoyer, Yoon Kim, Hannaneh Hajishirzi, Pang Wei Koh · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Training, objectives & optimization (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

56 physical pages

Before references / appendix

≈ 9.3 page-equivalents

Main figures / tables

6 / 4

Attached appendix figures / tables

33 / 13

Attached appendix span

≈ 42.91 page-equivalents

Before references / appendix: ≈ 16.61%References / intervening material: ≈ 6.77%Attached appendix: ≈ 76.63%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Source-bound correction: Join wrapped printed heading text using adjacent source lines; original heading start coordinates are unchanged.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 0.75 pp.	1.34%
2. Preliminaries	2	≈ 0.31 pp.	0.55%
3. RLER: Reinforcement Learning with Evolving Rubrics	2	≈ 1.22 pp.	2.18%
4. DR Tulu with Open Infrastructure and Training Recipe	4	≈ 0.7 pp.	1.25%
5. Experimental Results	4	≈ 2.38 pp.	4.25%
6. Analysis	7	≈ 2.51 pp.	4.48%
7. Related Works	9	≈ 0.18 pp.	0.32%
8. Conclusion	9	≈ 0.21 pp.	0.38%
Impact Statement	10	≈ 0.16 pp.	0.29%
Acknowledgments	10	≈ 0.1 pp.	0.18%
References	10	≈ 3.79 pp.	6.77%
Appendix	14	≈ 0.03 pp.	0.05%
Author Contributions	14	≈ 0.31 pp.	0.55%
A. Discussion and Future Work	14	≈ 1.02 pp.	1.82%
B. Related Work	15	≈ 1.07 pp.	1.91%
C. Problem Formulation for Deep Research	16	≈ 1.06 pp.	1.89%
D. Reinforcement Learning with Evolving Rubrics	17	≈ 5.18 pp.	9.25%
E. RLER Analysis and Toy Case Study	22	≈ 1.54 pp.	2.75%
F. Data Creation Details	24	≈ 4.55 pp.	8.13%
G. Training Details	28	≈ 1.98 pp.	3.54%
H. Experimental Details	30	≈ 5.56 pp.	9.93%
I. More Results and Analysis	36	≈ 11.46 pp.	20.46%
J. Analysis on Searched Tools and Domain Distributions	47	≈ 9.16 pp.	16.36%

All detected figure and table locations

Main text

• Figure 1 · p. 1
• Figure 2 · p. 3
• Table 1 · p. 5
• Figure 3 · p. 7
• Table 2 · p. 7
• Table 3 · p. 7
• Figure 4 · p. 8
• Figure 5 · p. 8
• Figure 6 · p. 8
• Table 4 · p. 9

Attached appendix

• Figure 7 · p. 14
• Table 5 · p. 16
• Figure 8 · p. 17
• Table 6 · p. 18
• Figure 9 · p. 18
• Figure 10 · p. 19
• Figure 11 · p. 20
• Figure 12 · p. 21
• Figure 13 · p. 23
• Figure 14 · p. 25
• Figure 15 · p. 26
• Figure 16 · p. 27
• Table 7 · p. 28
• Figure 17 · p. 29
• Table 8 · p. 30
• Table 9 · p. 31
• Table 10 · p. 31
• Figure 18 · p. 32
• Figure 19 · p. 33
• Figure 20 · p. 34
• Figure 21 · p. 35
• Figure 22 · p. 38
• Figure 23 · p. 39
• Figure 24 · p. 40
• Figure 25 · p. 41
• Table 11 · p. 42
• Table 12 · p. 42
• Figure 26 · p. 43
• Table 13 · p. 43
• Figure 27 · p. 44
• Table 14 · p. 44
• Figure 28 · p. 45
• Table 15 · p. 46
• Table 16 · p. 46
• Table 17 · p. 46
• Figure 29 · p. 47
• Figure 30 · p. 47
• Figure 31 · p. 48
• Figure 32 · p. 49
• Figure 33 · p. 50
• Figure 34 · p. 51
• Figure 35 · p. 52
• Figure 36 · p. 53
• Figure 37 · p. 54
• Figure 38 · p. 55
• Figure 39 · p. 56

Argument and organization

1. 1 Introduction; 2 Preliminaries; 3 RLER

   Motivates long-form research supervision and develops the evolving, search-grounded rubric mechanism.

2. 4 Open Infrastructure and Data; 5 Experimental Results

   Separates agent infrastructure and training from long-form benchmarks and an expert-domain application.

3. 6 Analysis; 7 Related Works; 8 Conclusion

   Tests data, rubric and judge choices before a broader context discussion and release-oriented conclusion.

Problem, claim and contribution

The introduction contrasts verifiable short answers with open-ended reports and positions adaptable rubric supervision as the missing training signal.

Abstract and Section 1, p.1 · Main paper

Argument and section order

The paper separates the reward mechanism, the infrastructure enabling asynchronous tool use and the evaluation, so implementation scale does not replace the causal training argument.

Sections 3-5, pp.3-6 · Main paper

Figures, captions and visual explanation

The training diagram follows a concrete query through rollouts, positive/negative rubric generation, buffer updates and policy scoring; the genetics plot defines four evidence dimensions in its caption.

Figure 2, p.3; Figure 3, p.7 · Main paper

Tables, comparisons and evidence

Analysis captions identify intermediate checkpoints and training-step budgets; the genetics caption discloses ten trials for two models versus three for costly proprietary systems.

Section 6 opening, Tables 2-3 and Figure 3, p.7; Figures 5-6, p.8 · Main paper

Appendix and reproducibility

The future-work discussion reframes evolving rubrics as adaptive verification, while a later variance section quantifies instability in repeated search-engine queries rather than assuming tools are fixed.

Appendix A, p.14; Appendix I.7, p.44 · Main paper

Clarity and writing conventions

Cost claims are tied to per-query tool/LM accounting on named datasets, and the analysis distinguishes increased training reward from downstream evaluation gains.

Section 5.2, p.6; Section 6.2, p.8 · Main paper

What to reuse—and what not to infer

Transferable lesson: For an agent-training system, give the reward loop a concrete example and disclose checkpoint, tool, repeated-trial and cost conditions next to each comparison.

Boundary: The cost headline depends on the task and API accounting, and search-tool variation limits exact repeatability; the expert-domain benchmark is not clinical deployment validation.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-oriented scan of the main argument through physical p.9, with focused physical-page text inspection on 1, 3, 5, 6, 7, 8, 9. Framing, section progression, mechanism/proof explanation, displays, comparison conditions and conclusions were inspected. This is not exhaustive technical reading, proof verification or reproduction.

Appendix / supplement

Appendix organization and main-text pointers were checked; focused text inspection of physical pages 14, 44. Remaining appendix proofs and details were not exhaustively read. No separately linked supplement inspected.

Visually inspected PDF pages

3, 7

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published proceedings PDF, PMLR volume 306, retrieved 2026-10-07 · 56 pages