Writing study: DD-Ranking: Rethinking the Evaluation of Dataset Distillation

NeurIPS overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← NeurIPS 2026 oral writing corpus

Zekai Li, Xinhao Zhong, Samir Khaki, Zhiyuan Liang, Yuhao Zhou, Mingjia Shi, Ziqiao Wang, Xuanlei Zhao, Wangbo Zhao, Ziheng Qin, Mengxuan Wu, Pengfei Zhou, Haonan Wang, David Junhao Zhang, Jia-Wei Liu, Shaobo Wang, Dai Liu, Linfeng Zhang, Guang Li, Kun Wang, Zheng Zhu, Zhiheng Ma, Joey Tianyi Zhou, Jiancheng Lv, Yaochu Jin, Peihao Wang, Kaipeng Zhang, Yiran Huang, Zhiwei Deng, Xindi Wu, George Cazenavette, Yuzhang Shang, Justin CUI, Jindong Gu, Qian Zheng, Hao Ye, Shuo Wang, Xiaobo Wang, Yan Yan, Angela Yao, Mike Zheng Shou, Tianlong Chen, Hakan Bilen, Baharan Mirzasoleiman, Manolis Kellis, Konstantinos N Plataniotis, Zhangyang "Atlas" Wang, Bo Zhao, Yang You, Kai Wang · Official NeurIPS 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Datasets, benchmarks & measurement (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

21 physical pages

Before references / appendix

≈ 9.24 page-equivalents

Main figures / tables

3 / 7

Attached appendix figures / tables

0 / 13

Attached appendix span

≈ 6.91 page-equivalents

Before references / appendix: ≈ 44%References / intervening material: ≈ 23.1%Attached appendix: ≈ 32.9%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Extraction notes: Table numbering discontinuity; see source note. Zero means no recognized caption, not independently verified absence.

Source-bound correction: Source-checked 2026-10-02: headings and reference/appendix boundaries are correct. The missing Table 20 label is a source feature, not a missed printed caption: page 20 contains four table panels (a)-(d), referenced as Table 20a-d in Appendix B, but no overall Table 20 caption. Retain the numbering-gap flag rather than invent a caption. Explicit-caption totals are not total visual-panel counts.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 Introduction	1	≈ 1.17 pp.	5.57%
2 Motivation	2	≈ 1.55 pp.	7.38%
3 DD-Ranking	4	≈ 1.19 pp.	5.67%
4 Results	5	≈ 3.8 pp.	18.1%
5 Related Works	9	≈ 0.51 pp.	2.43%
6 Conclusion and Future Work	9	≈ 0.4 pp.	1.9%
References	10	≈ 4.85 pp.	23.1%
A DD-Ranking Team	15	≈ 1.05 pp.	5%
B Additional Experiment Results	16	≈ 1.37 pp.	6.52%
C Additional Related Work	17	≈ 4.49 pp.	21.38%

All detected figure and table locations

Main text

• Table 1 · p. 2
• Figure 1 · p. 3
• Table 2 · p. 3
• Table 3 · p. 6
• Table 4 · p. 6
• Table 5 · p. 7
• Table 6 · p. 7
• Table 7 · p. 8
• Figure 2 · p. 8
• Figure 3 · p. 9

Attached appendix

• Table 8 · p. 16
• Table 9 · p. 16
• Table 10 · p. 16
• Table 11 · p. 17
• Table 12 · p. 17
• Table 13 · p. 18
• Table 14 · p. 18
• Table 15 · p. 18
• Table 16 · p. 18
• Table 17 · p. 19
• Table 18 · p. 19
• Table 19 · p. 19
• Table 21 · p. 21

Argument and organization

1. Problem and contribution

   The paper starts from a comparison-validity problem rather than a new distilled dataset. It argues that soft labels and post-evaluation training recipes can confound attribution of accuracy gains, then uses randomly sampled data under matched recipes as a reference.

2. Argument sequence

   A configuration audit and controlled motivation precede definitions of label-robust and augmentation-robust scores. Experiments then expose score components across datasets and images-per-class budgets before robustness checks across model architectures.

3. Evidence and verification

   CIFAR-10 and CIFAR-100 tables expose HLR, IOR and LRS separately within each images-per-class block, with metric directions and a stated weighting parameter. The captions retain the important exception that a soft-label method can remain strong rather than forcing a universal hard-label advantage.

Problem, claim and contribution

The paper starts from a comparison-validity problem rather than a new distilled dataset. It argues that soft labels and post-evaluation training recipes can confound attribution of accuracy gains, then uses randomly sampled data under matched recipes as a reference.

Introduction and motivation, pp. 1–4 · Main paper

Argument and section order

A configuration audit and controlled motivation precede definitions of label-robust and augmentation-robust scores. Experiments then expose score components across datasets and images-per-class budgets before robustness checks across model architectures.

Sections 2–5, pp. 2–9 · Main paper

Figures, captions and visual explanation

The opening comparison matrix partitions training hyperparameters from augmentation settings. Colored cells reveal protocol differences across methods at a glance. The text also names these differences, so the matrix acts as an audit of the comparison contract rather than a decorative heatmap.

Table 1, p. 2; rendered page inspected · Main paper

Tables, comparisons and evidence

CIFAR-10 and CIFAR-100 tables expose HLR, IOR and LRS separately within each images-per-class block, with metric directions and a stated weighting parameter. The captions retain the important exception that a soft-label method can remain strong rather than forcing a universal hard-label advantage.

Tables 3–4, p. 6; rendered page inspected · Main paper

Appendix and reproducibility

Additional tables vary the score's weighting parameter rather than treating one setting as self-evident. The extended related-work taxonomy records method families, helping distinguish a proposed metric from the many datasets and training recipes being re-evaluated.

Appendices B–C, pp. 16–21 · Main paper

Clarity and writing conventions

The most transferable pattern is to state the confound, the matched baseline and the score's interpretation before claiming a corrected ranking. General descriptions of unfairness should be scoped to the inspected evaluation protocols, not to every reported dataset-distillation result.

Sections 2–4 and conclusion, pp. 2–9 · Main paper

What to reuse—and what not to infer

Transferable lesson: When introducing an evaluation metric, first show the protocol mismatch, then expose the metric's components and sensitivity instead of presenting only its final ranking.

Boundary: LRS and ARS depend on their definitions, weighting and evaluation setup. The 2025 arXiv version is a public author source attached to a 2026 oral record, not evidence that the accepted revision is unchanged.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-02

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-oriented distributed manuscript scan of Sections 1–6, physical pp. 1–9: section sequence, substantive prose windows, captions, comparison contracts and conclusions. Not exhaustive technical verification.

Appendix / supplement

Writing-oriented distributed scan of Appendices A–C and supporting tables, physical pp. 15–21. Not independent proof checking; code and experiments were not executed.

Visually inspected PDF pages

2, 6

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Public author arXiv manuscript 2505.13300v3; not verified identical to the accepted or camera-ready OpenReview revision · 21 pages