Writing study: FRABench and UFEval: Unified Fine-grained Evaluation with Task and Aspect Generalization

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Shibo Hong, jiahao ying, Haiyuan Liang, Mengdi Zhang, Jun Kuang, Jiazheng Zhang, Yixin Cao · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Datasets, benchmarks & measurement resources (primary) · Learning & optimization algorithms.

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

72 physical pages

Before references / appendix

≈ 10.18 page-equivalents

Main figures / tables

5 / 7

Attached appendix figures / tables

18 / 68

Attached appendix span

≈ 56.24 page-equivalents

Before references / appendix: ≈ 14.14%References / intervening material: ≈ 7.75%Attached appendix: ≈ 78.11%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Extraction notes: Figure numbering discontinuity; see source note. Zero means no recognized caption, not independently verified absence.

Source-bound correction: UFEval: retain the source's actual numbering jump from Figure 22 on physical page 57 to Figure 36 on page 72. Figure 36 is visibly printed as 36, and page 71 explicitly references Figure 36. The intervening pages contain text and Tables 58-75, with no Figure 23-35 captions found in the raw bbox/text. Do not invent 13 figures or renumber Figure 36 to force a continuous series.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 1.1 pp.	1.53%
2 RELATED WORK	2	≈ 0.62 pp.	0.86%
3 METHODOLOGY	3	≈ 3.35 pp.	4.65%
4 EXPERIMENTS	6	≈ 4.03 pp.	5.6%
5 CONCLUSION, LIMITATION AND FUTURE WORK	10	≈ 0.37 pp.	0.51%
ACKNOWLEDGMENTS	11	≈ 0.07 pp.	0.1%
REFERENCES	11	≈ 5.58 pp.	7.75%
A ASPECT SOURCES AND DEFINITIONS	16	≈ 0.12 pp.	0.17%
B ANALYSIS OF THE FRABENCH	16	≈ 3.22 pp.	4.47%
C EXPERIMENT DETAIL	20	≈ 18.61 pp.	25.85%
D MORE EXPERIMENT RESULTS	38	≈ 11.39 pp.	15.82%
E HUMAN ANNOTATION	50	≈ 8 pp.	11.11%
F IMAGE UNDERSTADING & GENERATEION MODEL ALIGNMENT USING DPO	58	≈ 1 pp.	1.39%
G THE PROMPT TEMPLATE	59	≈ 4 pp.	5.56%
H QUALITATIVE EXAMPLES	63	≈ 8.51 pp.	11.82%
I THE QUALITATIVE COMPARISON FOR IMAGE GENERATION	71	≈ 0.15 pp.	0.21%
J THE USAGE OF LLM	71	≈ 1.23 pp.	1.71%

All detected figure and table locations

Main text

• Table 1 · p. 2
• Figure 1 · p. 4
• Figure 2 · p. 5
• Figure 3 · p. 6
• Table 2 · p. 6
• Figure 4 · p. 7
• Table 3 · p. 7
• Table 4 · p. 8
• Table 5 · p. 8
• Figure 5 · p. 9
• Table 6 · p. 10
• Table 7 · p. 10

Attached appendix

• Table 8 · p. 17
• Table 9 · p. 17
• Table 10 · p. 18
• Figure 6 · p. 18
• Figure 7 · p. 18
• Figure 8 · p. 18
• Figure 9 · p. 18
• Figure 10 · p. 19
• Figure 11 · p. 19
• Table 11 · p. 21
• Table 12 · p. 22
• Table 13 · p. 23
• Table 14 · p. 24
• Table 15 · p. 25
• Table 16 · p. 26
• Table 17 · p. 27
• Table 18 · p. 28
• Table 19 · p. 29
• Table 20 · p. 30
• Table 21 · p. 31
• Table 22 · p. 32
• Table 23 · p. 33
• Table 24 · p. 34
• Table 25 · p. 35
• Table 26 · p. 36
• Table 27 · p. 37
• Figure 12 · p. 38
• Table 28 · p. 38
• Figure 13 · p. 39
• Table 29 · p. 39
• Table 30 · p. 39
• Figure 14 · p. 40
• Table 31 · p. 40
• Figure 15 · p. 41
• Table 32 · p. 41
• Table 33 · p. 42
• Table 34 · p. 42
• Table 35 · p. 42
• Table 36 · p. 43
• Table 37 · p. 43
• Table 38 · p. 44
• Table 39 · p. 44
• Table 40 · p. 45
• Table 41 · p. 45
• Table 42 · p. 46
• Table 43 · p. 46
• Table 44 · p. 46
• Table 45 · p. 47
• Table 46 · p. 47
• Table 47 · p. 48
• Table 48 · p. 48
• Table 49 · p. 49
• Table 50 · p. 49
• Table 51 · p. 49
• Table 52 · p. 49
• Figure 16 · p. 50
• Figure 17 · p. 51
• Table 53 · p. 51
• Table 54 · p. 51
• Figure 18 · p. 52
• Table 55 · p. 52
• Table 56 · p. 53
• Table 57 · p. 54
• Figure 19 · p. 54
• Figure 20 · p. 55
• Figure 21 · p. 56
• Figure 22 · p. 57
• Table 58 · p. 59
• Table 59 · p. 59
• Table 60 · p. 60
• Table 61 · p. 60
• Table 62 · p. 61
• Table 63 · p. 61
• Table 64 · p. 61
• Table 65 · p. 62
• Table 66 · p. 62
• Table 67 · p. 63
• Table 68 · p. 64
• Table 69 · p. 65
• Table 70 · p. 66
• Table 71 · p. 67
• Table 72 · p. 68
• Table 73 · p. 69
• Table 74 · p. 70
• Table 75 · p. 71
• Figure 36 · p. 72

Argument and organization

1. Taxonomy creates the resource and evaluator

   The paper introduces universal versus task-specific criteria, then derives dataset construction, held-out splits and evaluator training.

2. Generalization, independent benchmarks, utility

   OOD task/aspect transfer is followed by public judge benchmarks, multi-task ablations and downstream DPO.

Problem, claim and contribution

A concrete distinction between transfer to new tasks and transfer to new criteria makes 'generalization' testable rather than an umbrella claim.

Sections 3.2.2 and 4.2; Table 2, pages 5–8 · Main paper

Argument and section order

Ablations study both task mixtures and criterion mixtures before downstream preference training, supporting the proposed cross-task synergy through several evidence types.

Sections 4.4–4.5; Tables 4–7 and Figure 5, pages 8–10 · Main paper

Figures, captions and visual explanation

The method figure combines two aspect taxonomies with one worked bus-image example; blue/orange arrows show why factual consistency and grammaticality can yield different rankings.

Figure 1, page 4 · Main paper

Tables, comparisons and evidence

Table 2 explicitly separates GPT-labeled and human-labeled tests, and shows the trained 7B evaluator still below GPT-4o on the selected human OOD tasks.

Table 2, page 6 · Main paper

Appendix and reproducibility

The unusually long appendix records definitions with source and target modality, mappings to benchmarks, annotation screens, per-aspect agreement and full templates; this is a useful audit trail, though navigation could be tighter.

Tables 13–27, 53–66; pages 23–37 and 50–62 · Main paper

Clarity and writing conventions

Dense radar plots summarize breadth but make individual differences difficult to read. Detailed tables permit verification; failed examples usefully expose over-sensitive harmfulness judgments and weak fidelity perception.

Figure 3, page 6; Tables 73–74, pages 69–70 · Main paper

What to reuse—and what not to infer

Transferable lesson: Define each generalization axis and label source before presenting results, and publish the operational rubric—not just the names of evaluation dimensions.

Boundary: Lexical ROUGE-L separation does not prove conceptual independence. Table 75 visibly contains mismatched copied feedback, so do not reuse that qualitative example or treat every appendix item as error-free.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

Pages 1–10, sections 1–5; reference list pages 11–16 not treated as substantive method evidence.

Appendix / supplement

Pages 16–72, A–J: aspect definitions/provenance, split and position-bias analysis, configurations, complete results, annotation reliability, DPO formulation, prompts, qualitative successes/failures, disclosures.

Visually inspected PDF pages

4, 6

• ICLR 2026 final proceedings · 72 pages