Writing study: Benchmark Health Index: A Systematic Framework for Benchmarking the Benchmarks of LLMs

NeurIPS overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← NeurIPS 2026 oral writing corpus

Longyuan Zhu, Hairan Hua, Linlin Miao, Wei Wang, Bing Zhao · Official NeurIPS 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Datasets, benchmarks & measurement (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

42 physical pages

Before references / appendix

≈ 17.11 page-equivalents

Main figures / tables

7 / 1

Attached appendix figures / tables

1 / 6

Attached appendix span

≈ 12.89 page-equivalents

Before references / appendix: ≈ 40.74%References / intervening material: ≈ 28.57%Attached appendix: ≈ 30.69%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Source-bound correction: Top-level heading and reference/appendix boundaries checked against source PDF text. Bibliography titles/author initials, numbered contribution/property bullets, diagram steps, equations and TOC entries are not substantive headings. Section spans are page-coordinate estimates, not ink area or prose-only length. This is an author manuscript mirror, not a verified accepted-version template. Source layout uses physical single-column y/page-height, including equation-heavy pages.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 2.76 pp.	6.57%
2. Related Work	4	≈ 0.98 pp.	2.33%
3. Methodology	5	≈ 5 pp.	11.9%
4. Data Collection	10	≈ 1.03 pp.	2.45%
5. Results	11	≈ 2.08 pp.	4.95%
6. Case Study	13	≈ 3.5 pp.	8.33%
7. Conclusion	17	≈ 0.18 pp.	0.43%
8. Limitations	17	≈ 0.16 pp.	0.38%
Acknowledgements	17	≈ 0.66 pp.	1.57%
References	18	≈ 12 pp.	28.57%
Appendix	30	≈ 0.04 pp.	0.1%
A. δ Threshold Sensitivity Analysis	30	≈ 0.66 pp.	1.57%
B. SDM Weighting and Normalization	30	≈ 1.64 pp.	3.9%
C. CV-based Weighting and Fusion	32	≈ 1.11 pp.	2.64%
D. CRITIC Objective Weighting Method	33	≈ 1.04 pp.	2.48%
E. Supplementary Details on Model Dropout Robustness	34	≈ 0.92 pp.	2.19%
F. Supplementary Details on Noise Robustness	35	≈ 0.74 pp.	1.76%
G. Supplementary Details on Ablation Study	36	≈ 6.74 pp.	16.05%

All detected figure and table locations

Main text

• Figure 1 · p. 2
• Figure 2 · p. 3
• Figure 3 · p. 4
• Figure 4 · p. 6
• Figure 5 · p. 8
• Table 1 · p. 9
• Figure 6 · p. 12
• Figure 7 · p. 14

Attached appendix

• Table 2 · p. 31
• Figure 8 · p. 31
• Table 3 · p. 35
• Table 4 · p. 35
• Table 5 · p. 36
• Table 6 · p. 36
• Table 7 · p. 37

Argument and organization

1. Introduction and framework

   Turns benchmark reliability into three separately named measurement questions.

2. Methodology and data collection

   Defines score construction before robustness and diagnosis of the ranked benchmark set.

3. Results, case studies and appendices

   Separates perturbation tests from domain-specific interpretations and extended audit tables.

Problem, claim and contribution

The motivation separates weak discrimination, saturation and influence. This makes the metric design legible as a response to three different problems, rather than another unexplained composite leaderboard score.

Introduction; Figure 2 · Main paper

Argument and section order

Definitions and data alignment precede robustness analysis and case studies. A reader can distinguish how the index is built from how it behaves and how a particular benchmark is diagnosed.

Sections 3–6 · Main paper

Figures, captions and visual explanation

Three parallel problem–solution columns converge on the composite index. Repeated colors tie each failure mode to its metric; the caption explains the rows. The diagram communicates rationale, not just computation.

Figure 2, PDF p. 3 · Main paper

Tables, comparisons and evidence

The comparison table exposes sub-metrics beside the aggregate score. Sensitivity and resampling analyses test whether a ranking depends on particular modeling choices; an aggregate alone would conceal that dependency.

Table 1, PDF p. 9; robustness appendices · Main paper

Appendix and reproducibility

The attached material carries threshold sensitivity, weighting derivations, dropout/noise protocols and expanded tables. These are useful audit routes for readers questioning the index construction; they are not interchangeable with independent benchmark validation.

Appendices A–G and full benchmark table · Main paper

Clarity and writing conventions

Stable names for the three axes aid navigation, while the limitations qualify reliance on non-standardized public reports. The manuscript’s promotional first-framework language should not be copied without an independent priority check.

Sections 3 and 8 · Main paper

What to reuse—and what not to infer

Transferable lesson: When proposing an evaluation index, show the failure-to-metric mapping and expose the component scores before presenting a single ranking.

Boundary: The inspected arXiv version has four named authors; the official program includes Wei Wang as a fifth. Treat this as a related public author revision, not the identical accepted copy. Its 42-page layout is not a NeurIPS page-budget example.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-02

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-focused scan across framing, method organization, captions, result presentation and conclusion; title and author overlap checked against the official program. Selected physical pages inspected as rendered images.

Appendix / supplement

Attached appendix headings, reproduction routes and representative supporting prose/captions scanned. Long tables, every proof and every appendix page were not individually validated.

Visually inspected PDF pages

3, 9

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Public author manuscript arXiv:2602.11674v1; not verified identical to the accepted OpenReview revision · 42 pages