Skip to content
Computational Pathology Reading Roadmap

Writing study: Position: There are futures that benchmark-driven AI cannot see

← ICML 2026 oral writing corpus

Sobhan Lotfi, Ava Iranmanesh, Lachin Naghashyar, Ali Shirali, Fateme Nateghi Haredasht, Sanmi Koyejo, Philip Torr, Yong Suk Lee, Fazl Barez, Joel Lehman, Peter Norvig, Arvind Narayanan · Official ICML 2026 oral record

Figure, table and section ledger

Editorial contribution type: Position & research agenda (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF
13 physical pages
Before references / appendix
≈ 9.04 page-equivalents
Main figures / tables
0 / 0
Attached appendix figures / tables
0 / 0
Attached appendix span
No boundary detected; not assumed zero
Before references / appendix: ≈ 69.54%References / intervening material: ≈ 30.46%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Extraction notes: No attached appendix boundary detected. Zero means no recognized caption, not independently verified absence.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level headingStarts on PDF pageEstimated spanShare of entire PDF
1. Introduction1≈ 0.95 pp.7.31%
2. Background: Exaptation and AI History2≈ 1.36 pp.10.46%
3. Present Diagnosis: The Tyranny of3≈ 2.05 pp.15.77%
4. The Such-That Problem5≈ 1.99 pp.15.31%
5. Proposals7≈ 1.41 pp.10.85%
6. Alternative Views9≈ 0.83 pp.6.38%
7. Discussion9≈ 0.17 pp.1.31%
References10≈ 3.96 pp.30.46%
All detected figure and table locations

Main text

    No recognized caption labels.

    Attached appendix

      No recognized caption labels.

      Argument and organization

      1. 1 Introduction; 2 Background

        Defines the exaptation analogy and contrasts capability evaluation with normative such-that goals.

      2. 3 Present Diagnosis; 4 The Such-That Problem

        Separates community composition, agenda setting and limits of evaluation before arguing why normative objectives differ.

      3. 5 Proposals; 6 Alternative Views; 7 Discussion

        Offers institutional interventions, responds to five objections and closes with a portfolio-hedging argument.

      Problem, claim and contribution

      The thesis is diversification of evaluation, not abolition of benchmarks; the introduction makes this contrast before the historical examples.

      Sections 1–2, physical pp.1–2 · Main paper

      Argument and section order

      Diagnosis, proposed interventions and alternative views have distinct sections, allowing a reader to challenge premises separately from recommendations.

      Sections 3–7, pp.3–9 · Main paper

      Figures, captions and visual explanation

      The inspected pages use no result charts: descriptive subsection labels and bold objection names carry the argument’s navigation. A benchmark leaderboard would not supply evidence for the central normative claim.

      Sections 5–6, pp.7,9; rasters inspected · Main paper

      Tables, comparisons and evidence

      Historical cases and cited research support the argument, but the invisibility objection explicitly concedes that specific lost-innovation counterfactuals cannot be observed.

      Section 6, p.9 · Main paper

      Appendix and reproducibility

      No appendix is included in this source. The objections and qualifications remain in the main paper rather than being deferred behind the references.

      Sections 6–7, p.9; references begin p.10 · Main paper

      Clarity and writing conventions

      The protection objection proposes time-limited support and ex-post tracking, and distinguishes plural evaluation from absence of accountability.

      Section 6, p.9 · Main paper

      What to reuse—and what not to infer

      Transferable lesson: In a position paper, separate diagnosis, actionable proposals and serious objections; state what evidence cannot resolve the argument.

      Boundary: Historical analogies and cited correlations do not causally establish that benchmark culture suppressed particular innovations or that the proposed governance interventions will work. This is a position argument, not an experimental comparison or official conference policy.

      Apply this lesson using the argument and evidence worksheet.

      Inspection record and sources

      Checked
      2026-10-07
      Reading status
      Writing-focused manuscript scan + selected visuals inspected
      Main text
      Writing-focused scan across physical PDF pp.1–9, covering framing, section progression, definitions, method/evidence explanations, captions and scope. Physical pages extracted individually to avoid form-feed pagination errors. Selected full-page rasters inspected; not exhaustive line-by-line technical verification or experimental reproduction.
      Appendix / supplement
      Main argument pp.1–9 and reference boundary p.10 checked. This 13-page proceedings PDF has no separate appendix; historical citations were not independently re-audited.
      Visually inspected PDF pages
      7, 9

      This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.