Writing study: P-GenRM: Personalized Generative Reward Model with Test-time User-based Scaling

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Pinyi Zhang, Ting-En Lin, Yuchuan Wu, Jingyang Chen, Zongqi Wang, Hua Yang, Bing Zhao, Fei Huang, Yongbin Li, Kai Zhang · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Learning & optimization algorithms (primary) · Architecture & representation design.

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

29 physical pages

Before references / appendix

≈ 10.18 page-equivalents

Main figures / tables

4 / 5

Attached appendix figures / tables

11 / 10

Attached appendix span

≈ 15.89 page-equivalents

Before references / appendix: ≈ 35.1%References / intervening material: ≈ 10.1%Attached appendix: ≈ 54.79%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 1.48 pp.	5.1%
2 RELATED WORKS	3	≈ 0.51 pp.	1.76%
3 PROBLEM FORMULATION	3	≈ 0.79 pp.	2.72%
4 METHODOLOGY	4	≈ 2.79 pp.	9.62%
5 EXPERIMENTS	7	≈ 3.59 pp.	12.38%
6 CONCLUSION	10	≈ 0.32 pp.	1.1%
ACKNOWLEDGMENTS	11	≈ 0.07 pp.	0.24%
REFERENCES	11	≈ 2.92 pp.	10.07%
A APPENDIX	14	≈ 7.32 pp.	25.24%
B DATASETS AND BASELINES	21	≈ 1.68 pp.	5.79%
C PROMPTS	23	≈ 6.89 pp.	23.76%

All detected figure and table locations

Main text

• Figure 1 · p. 2
• Figure 2 · p. 4
• Table 1 · p. 8
• Table 2 · p. 8
• Table 3 · p. 8
• Figure 3 · p. 9
• Table 4 · p. 9
• Figure 4 · p. 10
• Table 5 · p. 10

Attached appendix

• Table 6 · p. 14
• Table 7 · p. 15
• Figure 5 · p. 15
• Table 8 · p. 16
• Table 9 · p. 16
• Figure 6 · p. 16
• Table 10 · p. 17
• Figure 7 · p. 17
• Figure 8 · p. 18
• Table 11 · p. 19
• Table 12 · p. 20
• Table 13 · p. 20
• Table 14 · p. 21
• Table 15 · p. 21
• Figure 9 · p. 23
• Figure 10 · p. 24
• Figure 11 · p. 25
• Figure 12 · p. 26
• Figure 13 · p. 27
• Figure 14 · p. 28
• Figure 15 · p. 29

Argument and organization

1. Two personalization failures → evaluation chains

   Maps static rubrics and cold-start transfer to separate design choices.

2. Three training stages → two scaling levels

   Separates model learning from inference-time aggregation.

3. Preference accuracy → prototypes → OOD and policy evaluation

   Tests both the reward model and its proposed downstream utility.

Problem, claim and contribution

The introduction distinguishes scenario-dependent preferences from sparse-history generalization, allowing persona-conditioned rubrics and prototype transfer to answer different questions.

Section 1, pp. 1–2 · Main paper

Argument and section order

SFT, mixed-reward GRPO and hard-negative curriculum are named separately; the curriculum explicitly removes process reward, avoiding an ambiguous single training recipe.

Section 4.1, pp. 4–5 · Main paper

Figures, captions and visual explanation

The overview separates an offline prototype strip from the test-time user path and draws individual and neighbor samples as distinct stacks before aggregation; the large evaluation-chain inset is explanatory but text-dense.

Figure 1, p. 2 · Main paper

Tables, comparisons and evidence

The main comparison groups prompting, scalar fine-tuning, personalized models and scaling variants, labels unreported results with dashes and states mean/standard error over five runs.

Table 1, p. 8 · Main paper

Appendix and reproducibility

Prototype-wise macro accuracy, actual prompts, sample-count sensitivity and dataset-wide timing supplement the headline means. The timing protocol discloses eight A100s for local models and API concurrency for proprietary judges.

Appendices A.5, A.9, A.11 and C, pp. 15–29 · Main paper

Clarity and writing conventions

The cold-start experiment explicitly explains that its reference ranking is generated by model judges; this provenance should remain attached to any claim of out-of-distribution generalization.

Section 5.4, p. 10; Appendix A.7, p. 18 · Main paper

What to reuse—and what not to infer

Transferable lesson: Separate training improvements from test-time sampling gains and show whose preferences, which judges and what compute define each evaluation.

Boundary: LaMP-QA agreement is with aggregated LLM judgments, not new human preference labels. Macro accuracy over learned prototypes is not a demographic fairness guarantee, and three observed history examples are an empirical operating point rather than a universal minimum for personalization.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

Pages 1–10, Sections 1–6, full training, prototype refinement, test-time aggregation and all evaluations.

Appendix / supplement

Pages 14–29, Appendices A–C, including every analysis, policy experiment, algorithm, baseline/dataset description and prompt.

Visually inspected PDF pages

2, 8, 19

• ICLR 2026 official proceedings PDF, retrieved 2026-09-19 · 29 pages