Writing study: AdAEM: An Adaptively and Automated Extensible Measurement of LLMs' Value Difference
← ICLR 2026 oral writing corpus
Jing Yao, Shitong Duan, Xiaoyuan Yi, Dongkuan Xu, Peng Zhang, Tun Lu, Ning Gu, Zhicheng Dou, Xing Xie · Official ICLR 2026 oral record
Figure, table and section ledger
Editorial contribution type: Datasets, benchmarks & measurement resources (primary) · Learning & optimization algorithms.
Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.
- Canonical PDF
- 49 physical pages
- Before references / appendix
- ≈ 10.62 page-equivalents
- Main figures / tables
- 9 / 1
- Attached appendix figures / tables
- 12 / 17
- Attached appendix span
- ≈ 27.89 page-equivalents
Section-by-section lengths and source pages
The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.
| Top-level heading | Starts on PDF page | Estimated span | Share of entire PDF |
|---|---|---|---|
| 1 INTRODUCTION | 1 | ≈ 1.04 pp. | 2.12% |
| 2 RELATED WORKS | 2 | ≈ 1.04 pp. | 2.12% |
| 3 METHODOLOGY | 3 | ≈ 2.69 pp. | 5.49% |
| 4 ADAEM ANALYSIS | 6 | ≈ 3.15 pp. | 6.43% |
| 5 VALUE EVALUATION WITH ADAEM | 9 | ≈ 0.9 pp. | 1.84% |
| 6 CONCLUSION AND FUTURE WORK | 10 | ≈ 0.15 pp. | 0.31% |
| ETHICS STATEMENT | 10 | ≈ 0.81 pp. | 1.65% |
| REPRODUCIBILITY STATEMENT | 11 | ≈ 0.21 pp. | 0.43% |
| REFERENCES | 11 | ≈ 10.48 pp. | 21.39% |
| A DISCUSSION ON LLMS’ VALUE | 22 | ≈ 0.63 pp. | 1.29% |
| B DETAILS OF DATASET CONSTRUCTION | 22 | ≈ 2.37 pp. | 4.84% |
| C EXPERIMENTAL DETAILS | 25 | ≈ 10.47 pp. | 21.37% |
| D DETAILED DERIVATION | 35 | ≈ 2.04 pp. | 4.16% |
| E ADDITIONAL RESULTS | 37 | ≈ 4.49 pp. | 9.16% |
| F LLM USAGE | 42 | ≈ 0.15 pp. | 0.31% |
| G ADDITIONAL DISCUSSION | 42 | ≈ 0.95 pp. | 1.94% |
| H LIMITATIONS | 43 | ≈ 0.9 pp. | 1.84% |
| I DISCUSSION ON THE MATHEMATICAL APPROXIMATION OF ADAEM | 44 | ≈ 1.31 pp. | 2.67% |
| J ADAEM FRAMEWORK ON MORAL FOUNDATION THEORY | 45 | ≈ 1.96 pp. | 4% |
| K ANALYSIS ON HYPERPARAMETER ROBUSTNESS | 47 | ≈ 2.62 pp. | 5.35% |
All detected figure and table locations
Main text
Attached appendix
- Figure 10 · p. 23
- Figure 11 · p. 24
- Table 2 · p. 24
- Figure 12 · p. 24
- Table 3 · p. 25
- Table 4 · p. 26
- Table 5 · p. 27
- Table 6 · p. 30
- Table 7 · p. 32
- Table 8 · p. 32
- Table 9 · p. 33
- Table 10 · p. 34
- Table 11 · p. 35
- Table 12 · p. 35
- Figure 13 · p. 38
- Figure 14 · p. 39
- Figure 15 · p. 39
- Figure 16 · p. 40
- Figure 17 · p. 40
- Table 13 · p. 41
- Figure 18 · p. 41
- Table 14 · p. 45
- Figure 19 · p. 46
- Table 15 · p. 46
- Table 16 · p. 47
- Table 17 · p. 47
- Figure 20 · p. 48
- Figure 21 · p. 48
- Table 18 · p. 49
Argument and organization
- Informativeness rather than generic agreement
Uses paired questions to motivate a dynamic evaluation objective.
- Objective, alternating refinement and exploration
Separates scoring from the search policy and final value aggregation.
- Quality, validity, reliability and extensibility
Tests different properties of the measurement instrument before model comparisons.
Problem, claim and contribution
A generic question and a contextualized variant make the motivation concrete before mathematical notation is introduced.
Figure 1 and Section 1, p. 2 · Main paper
Argument and section order
Separating instrument validity and reliability from downstream model rankings is a useful benchmark-paper structure.
Sections 4–5, pp. 6–10 · Main paper
Figures, captions and visual explanation
The central diagram shows the question-refinement loop and resulting evaluation set in one view, but the tiny objective competes with example responses for attention.
Figure 2, p. 3 · Main paper
Tables, comparisons and evidence
Figure 9 prints scores on every bar, but its ten panels use different truncated horizontal scales; bar lengths cannot safely be compared between value dimensions.
Figure 9, p. 10 · Main paper
Appendix and reproducibility
The appendix provides model versions, prompts, human-annotation compensation, a notation table and ablations. These are more useful for replication than the abbreviated main algorithm alone.
Appendices B–C and K, pp. 22–35, 47–49 · Main paper
Clarity and writing conventions
The conceptual objective and practical reward need sharper separation: Appendix C.3 uses heuristic scores and a negative disentanglement term, whereas the main objective presents a positive divergence term.
Equation 1, p. 4; Appendix C.3, pp. 29–30; Appendix I, pp. 44–45 · Main paper
What to reuse—and what not to infer
Transferable lesson: For a new evaluation instrument, organize evidence around what it measures, whether it responds to controlled interventions, and how stable its results are—not only whether it separates models.
Boundary: Greater disagreement is not proof of more accurate measurement of intrinsic values. Model nationality is not a causal explanation. The stated JS-versus-total-variation bound and zero-intersection denominators require independent checking; do not inherit the convergence guarantee for the heuristic implementation.
Apply this lesson using the argument and evidence worksheet.
Inspection record and sources
- Checked
- 2026-09-19
- Reading status
- Full text + selected visuals inspected
- Main text
- PDF pp. 1–11, Sections 1–6, ethics and reproducibility statements
- Appendix / supplement
- PDF pp. 22–49, Appendices A–K including all prompt listings and derivations
- Visually inspected PDF pages
- 3, 10