Skip to content
Computational Pathology Reading Roadmap

Evaluation and reproducibility

Evaluation is the process that connects an experimental result to a claim. A model can be implemented correctly and still produce invalid evidence through leakage, a misaligned metric, repeated test-set selection, unpaired uncertainty, hidden subgroups, or an irreproducible pipeline.

Latest literature discovery audit: 2 October 2026. Existing paper reviews retain their individual source-check dates.

Prerequisites

Section titled “Prerequisites”

Complete sampling, estimation, confidence intervals, and hypothesis tests in mathematics, probability, and statistics. You should understand the complete training/selection pipeline of at least one method. This chapter should be applied during every other route, not saved until the end.

Learning objectives

Section titled “Learning objectives”

After this chapter, you should be able to:

  • turn a scientific statement into a population, estimand, design, and analysis;
  • choose split units and model-selection boundaries that block information leakage;
  • select discrimination, calibration, ranking, regression, segmentation, retrieval, survival, or generative metrics that match the decision;
  • quantify paired finite-sample and run-to-run uncertainty;
  • distinguish aleatoric, epistemic, distributional, and decision uncertainty;
  • stress-test subgroups and specified distribution shifts;
  • construct fair baselines, ablations, and compute-matched comparisons;
  • package code, data identity, environment, configuration, logs, and artifacts so another researcher can inspect the result;
  • write a model or dataset card whose limitations are operationally useful.

Start with a claim–estimand–design contract

Section titled “Start with a claim–estimand–design contract”

Before training, complete this table:

Contract element Required statement
intended use the user, decision, timing, and consequence of the output
population cases, domains, and time period to which the claim should generalize
sample inclusion/exclusion, collection mechanism, missingness, and label process
independent unit the unit that can be considered independently sampled
input and target information available at prediction time and what the label means
estimand the population performance difference or other quantity to estimate
intervention the component or training condition being compared
selection protocol preprocessing, tuning, checkpoint, threshold, and stopping rules
test protocol untouched split/domain, metrics, subgroups, and uncertainty procedure
deployment boundary conditions under which the evidence no longer applies

An estimand is more precise than “model performance.” An example is: “the difference in patient-level log loss between models A and B on first biopsies from hospitals not used for training, after both receive equal tuning.”

Split design and leakage

Section titled “Split design and leakage”

The independence unit

Section titled “The independence unit”

Split at the highest unit that carries shared information relevant to generalization: patient before case, case before slide, document before chunk, video before frame, or speaker before utterance. Near duplicates, augmentations, serial samples, family relationships, templates, and pretrained benchmark contamination can cross nominal row boundaries.

Which split answers which question?

Section titled “Which split answers which question?”
Split Target claim Main limitation
random independent-unit split new examples from the same mixture hides domain and temporal shift
grouped split new groups such as patients or speakers may still share acquisition site or time
temporal split future cases labels, practices, or censoring may differ
leave-site-out a new institution/device/domain few sites give uncertain domain-level inference
external cohort transport to a separately collected population protocol differences may confound the source of change
cross-dataset benchmark broad robustness task and label definitions may not match

Preprocessing learned from data belongs inside the training boundary: imputation, normalization, vocabulary/tokenizer learning when applicable, feature selection, PCA, oversampling, representation learning, augmentation policy selection, calibration, and threshold choice. Deduplicate before splitting, but retain an audit trail of the rule and removed groups.

Model selection and the final test

Section titled “Model selection and the final test”

A validation set or inner cross-validation selects hyperparameters, architecture, checkpoint, ensemble, threshold, and preprocessing. The final test is evaluated after those decisions are frozen. Looking repeatedly at test results and adapting the method turns that set into development data.

Nested cross-validation estimates a complete selection procedure in small samples. Repeated outer folds are not independent datasets; use paired, hierarchy-aware uncertainty rather than treating every fold score as independent.

Metrics must match outputs and decisions

Section titled “Metrics must match outputs and decisions”

Classification

Section titled “Classification”

From a thresholded binary classifier, sensitivity, specificity, precision, and negative predictive value answer different conditional questions. Precision and predictive values change with prevalence. Accuracy can hide failure on a rare class.

Threshold-free ranking metrics also differ:

  • ROC AUC is the probability a randomly chosen positive ranks above a negative under the sampled population;
  • precision–recall curves emphasize positive-class retrieval and depend on prevalence;
  • average precision is a ranking summary, not performance at a deployable threshold.

For probabilities, report proper scoring rules such as log loss or Brier score and calibration. Expected calibration error depends on bins and can hide local or subgroup miscalibration. Inspect reliability diagrams and calibration-in-the-large/slope. Calibrate using held-out development data, never the final test.

Multiclass, multilabel, and ordinal tasks

Section titled “Multiclass, multilabel, and ordinal tasks”

State macro, micro, weighted, or per-class averaging. Macro averaging weights classes equally; micro averaging weights individual decisions. Multilabel tasks require instance versus label aggregation and a threshold policy. Ordinal targets should penalize distance appropriately and report clinically meaningful disagreement.

Regression and counts

Section titled “Regression and counts”

MAE estimates absolute-error performance and is robust relative to squared error; MSE emphasizes large errors; R² is relative to a reference and can be negative on test data. Inspect residuals across target range and groups. For counts, skewed targets, or heteroscedastic noise, use a likelihood and intervals appropriate to the outcome.

Segmentation and detection

Section titled “Segmentation and detection”

Dice and intersection-over-union couple overlap to object size and do not describe boundary error. Add surface distance or boundary metrics when shape matters and define empty-mask handling. Detection average precision depends on IoU thresholds, matching, class averaging, and confidence ranking. Report object-level sensitivity and false positives at an operational unit when those drive use.

Retrieval and representation

Section titled “Retrieval and representation”

Recall@k, precision@k, mean reciprocal rank, and mean average precision depend on the candidate pool and relevance labels. Remove duplicates and same-subject matches when retrieval should generalize across subjects. A linear probe needs an independently tuned readout and fixed encoder checkpoint.

Survival and time-to-event outcomes

Section titled “Survival and time-to-event outcomes”

Censoring requires time-aware methods. A concordance index measures ranking, not calibration or absolute risk. Time-dependent AUC, Brier score, and calibration require clear horizons and censoring assumptions. Randomly classifying censored cases at an arbitrary cutoff changes the scientific target.

Generative systems

Section titled “Generative systems”

Likelihood, FID, precision/recall, human preference, conditional correctness, memorization, and inference cost cover different properties. Declare feature extractor, sample count, reference data, prompt/condition distribution, rater protocol, and duplicate checks. See generative and representation learning for mechanism-specific interpretation.

Uncertainty

Section titled “Uncertainty”

Sources of uncertainty

Section titled “Sources of uncertainty”
  • aleatoric: outcome variability remaining under the variables/model;
  • epistemic: uncertainty about functions or parameters from finite information;
  • sampling: variation in the test estimate across possible sampled cases;
  • optimization: variation across initialization, order, and stochastic training;
  • distributional: uncertainty about whether future data follow the evaluated process;
  • decision: uncertainty in utilities, thresholds, and consequences.

Predictive entropy mixes several sources. A softmax vector is not a complete uncertainty estimate.

Paired intervals

Section titled “Paired intervals”

When two models predict the same test cases, compute a paired difference per resample rather than two unrelated intervals. Bootstrap at the independent unit; stratify or cluster when the sampling design requires it. For a small number of sites, distinguish patient-level uncertainty conditional on those sites from uncertainty over future sites.

Across stochastic training runs, report all prespecified seeds and summarize their distribution. Do not inflate the nominal sample size by treating batches, epochs, or checkpoint evaluations as independent.

Predictive uncertainty methods

Section titled “Predictive uncertainty methods”

Deep ensembles average independently trained predictors and provide a strong practical baseline, but their diversity depends on training and they do not represent a uniquely defined posterior. Bayesian approximations, Monte Carlo dropout, evidential methods, and test-time augmentation encode different uncertainties and need coverage checks.

Conformal prediction calibrates set or interval coverage under exchangeability (or a method-specific weaker condition). Marginal coverage does not guarantee coverage for every subgroup or individual, and distribution shift can invalidate it. Report set size/interval width, conditional diagnostics, and the calibration population.

Statistical comparison

Section titled “Statistical comparison”

Start with effect size and interval. A null-hypothesis test addresses compatibility with a specified null under assumptions; it does not give the probability that the hypothesis or paper is true.

For model comparisons:

  • pair predictions or resamples on the same independent units;
  • pre-specify the primary model, metric, horizon, and subgroup;
  • account for multiplicity when many claims are promoted;
  • distinguish confirmatory tests from exploration;
  • report practical equivalence or non-inferiority margins when those match the claim;
  • avoid selecting the best of many experiments and testing it as if it were pre-specified.

Cross-validation comparison procedures have dependence. A table of fold-wise mean ± standard deviation is not automatically a valid population confidence interval.

Calibration, abstention, and decisions

Section titled “Calibration, abstention, and decisions”

Calibration asks whether predicted probabilities match observed frequencies under a population and time horizon. It does not establish discrimination. Temperature scaling can calibrate multiclass logits with a development set; isotonic and other flexible methods need enough data.

An abstaining or selective model requires a coverage–risk curve and a definition of who handles rejected cases. Evaluate whether abstention disproportionately removes particular groups. Decision-curve or expected-utility analysis requires explicit benefit/cost assumptions; do not infer clinical utility from AUC alone.

Distribution shift and subgroup evaluation

Section titled “Distribution shift and subgroup evaluation”

Name the shift and its mechanism:

  • covariate or acquisition shift;
  • prevalence/label shift;
  • conditional or concept shift;
  • site, device, demographic, temporal, or policy shift;
  • missing modality or corrupted input;
  • open-set classes and out-of-distribution inputs.

WILDS provides a benchmark framework for real-world distribution shifts, but strong performance on its datasets is not universal robustness. For each expected shift, define a stress set before testing, preserve the true unit, and report both aggregate and per-domain results with uncertainty.

Subgroup analysis needs justified groups, enough samples, intersectional checks, and the same label quality. A performance difference can arise from prevalence, measurement, spectrum, sample size, or true conditional error. Small noisy subgroup estimates should not be hidden or overinterpreted.

Out-of-distribution detection must specify which out-distribution and operational action are relevant. Evaluating on a visually obvious synthetic corruption does not show detection of a new hospital, disease, or adversarial input. The maximum-softmax baseline is a useful minimum control, not a calibrated guarantee: name the in/out populations, preserve an untouched threshold-selection set, report AUROC and precision–recall under realistic prevalence, and evaluate the action taken after detection.

Baselines, ablations, and falsification

Section titled “Baselines, ablations, and falsification”

Baselines

Section titled “Baselines”

Include:

  1. a prevalence, mean, random, or persistence baseline;
  2. a simple interpretable model;
  3. the strongest relevant established model;
  4. an information-matched version without the proposed component;
  5. an oracle only when clearly labelled as an unattainable upper reference.

Give every learned baseline the same data, labels, pretrained information policy, model-selection trials, stopping access, seeds, and compute boundary. Reproduce from official code when possible and document deviations.

Ablations

Section titled “Ablations”

An ablation changes one causal component while preserving everything else. Removing a module while also changing parameter count, receptive field, training steps, or augmentation does not isolate the module. Add parameter-matched and compute-matched controls when the claim requires them.

Negative results, failed runs, sensitivity curves, and interaction ablations reveal where a mechanism works. A long table of toggles is not a substitute for a hypothesis.

Robustness checks

Section titled “Robustness checks”

Stress label noise, missing input, resolution, sample size, class prevalence, domain, seed, hyperparameter range, and compute. Distinguish robustness known before analysis from a post hoc search for a favourable view.

Reproducibility and artifact identity

Section titled “Reproducibility and artifact identity”

Reproducibility has several levels:

Level Question
computational repeatability can the same code/data/environment reproduce the run within declared tolerance?
independent reproducibility can another implementation and team obtain the conclusion?
replicability across data does the claim hold in a new sample or domain?
auditability can a reader reconstruct exactly what information and choices produced the result?

A minimum experiment bundle records:

claim and protocol
├── code commit and dirty-worktree state
├── immutable configuration and command
├── data source, licence, version, checksums, and split IDs
├── environment lockfile, container/OS, framework, drivers, and hardware
├── random seeds and determinism settings
├── logs for every run, including failures
├── checkpoints and prediction files keyed by example ID
├── metric code and bootstrap/resampling indices
└── tables/figures generated from saved results, not hand-transcribed

Deterministic algorithms can reduce one source of variation but may cost speed and do not guarantee identical results across releases or hardware. Follow the official PyTorch reproducibility notes, record backend settings, and test whether conclusions survive ordinary run-to-run variation.

Resource and systems reporting

Section titled “Resource and systems reporting”

Report model parameters, trainable parameters, input size, examples/tokens processed, optimizer steps, effective batch, tuning trials, failed trials, precision, hardware, peak memory, wall time, and inference latency/throughput. State whether preprocessing, pretraining, data curation, and hyperparameter search are inside the boundary.

Accuracy per parameter is not accuracy per joule or per dollar. Hardware utilization, parallelism, region, queue time, and amortized pretraining matter. Do not trigger paid cloud workflows merely to reproduce a tutorial; provide a small local configuration and an explicit scaling estimate.

Documentation and governance

Section titled “Documentation and governance”

Datasheets document dataset motivation, composition, collection, preprocessing, licensing, maintenance, and known limitations. Model cards document intended use, evaluation context, groups, ethical considerations, and limitations. They complement technical reports; they do not certify safety.

The NIST AI Risk Management Framework organizes governance, mapping, measurement, and management of AI risks. It is a voluntary risk framework, not a benchmark score or regulatory approval. Apply the requirements of the actual jurisdiction and institution separately.

Staged primary-source and standards ladder

Section titled “Staged primary-source and standards ladder”
Order Topic Primary paper, official guide, or standard · year · status Read it for
1 resampling Bootstrap Methods: Another Look at the Jackknife — 1979 · Annals of Statistics empirical resampling logic and its assumptions
2 multi-dataset comparison Statistical Comparisons of Classifiers over Multiple Data Sets — 2006 · JMLR paired ranks, multiple classifiers, and comparison design
3 calibration On Calibration of Modern Neural Networks — 2017 · ICML reliability diagrams, ECE, and temperature scaling
4 predictive uncertainty Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles — 2017 · NeurIPS independently trained ensembles as an uncertainty baseline
5 OOD baseline A Baseline for Detecting Misclassified and Out-of-Distribution Examples — 2017 · ICLR maximum-softmax confidence, its useful baseline role, and its limitations
6 generative metric GANs Trained by a Two Time-Scale Update Rule — 2017 · NeurIPS the original FID definition and dependence on feature distribution and sample protocol
7 subgroup fairness Gender Shades — 2018 · FAT* intersectional slices, subgroup sample balance, and commercial-system auditing
8 model documentation Model Cards for Model Reporting — 2019 · FAT* intended use, disaggregated evaluation, and limitations
9 reproducible research Improving Reproducibility in Machine Learning Research — 2021 · JMLR checklists, code submission, and evidence from a conference-scale intervention
10 dataset documentation Datasheets for Datasets — 2021 · Communications of the ACM a lifecycle-oriented dataset questionnaire
11 distribution shift WILDS: A Benchmark of in-the-Wild Distribution Shifts — 2021 · ICML realistic shift definitions, datasets, and evaluation
12 calibrated sets A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification — 2023 · Foundations and Trends in Machine Learning exchangeability, marginal coverage, and practical conformal methods
13 risk management AI Risk Management Framework 1.0 — 2023 · NIST official framework; stable landing page govern–map–measure–manage risk functions
14 implementation control PyTorch Reproducibility — official, versioned framework guidance randomness sources, deterministic operations, and platform limits

Controlled lab

Section titled “Controlled lab”

Create a grouped binary dataset with repeated observations per subject, imbalance, a site feature, and a held-out shifted site. Keep a hidden copy of the data-generating process so the failure is measurable.

  1. Compare a row-random split with a subject-grouped split.
  2. Fit preprocessing outside the split once and correctly inside the pipeline once.
  3. Tune a linear model and a neural model with equal trials; freeze the final choice.
  4. Report ROC AUC, average precision, log loss, Brier score, a reliability diagram, and an operational threshold table.
  5. Bootstrap the paired model difference at row level and subject level; compare interval widths and coverage across repeated synthetic datasets.
  6. Calibrate on development data and evaluate the shifted site without refitting.
  7. Report each site/subgroup, an ensemble uncertainty baseline, and a conformal set with its empirical coverage and size.
  8. Remove the proposed component with parameter- and compute-matched controls.
  9. Reproduce the table from a fresh environment using saved split IDs and predictions.

The expected result is not merely a high score; it is an audit showing how leakage, resampling unit, calibration, and shift alter the conclusion.

Common failure modes

Section titled “Common failure modes”
  • Defining the split after inspecting outcomes or model errors.
  • Splitting correlated observations rather than independent groups.
  • Reusing the final test for architecture, prompt, checkpoint, or threshold selection.
  • Reporting a metric whose prevalence, aggregation, or empty-case rule is unspecified.
  • Treating cross-validation folds, checkpoints, or tiles as independent replicates.
  • Showing separate confidence intervals instead of a paired difference.
  • Conflating predictive entropy, calibration, and epistemic uncertainty.
  • Claiming conformal subgroup coverage from marginal coverage alone.
  • Reporting subgroup point estimates without sample counts and intervals.
  • Calling synthetic corruption robustness “out-of-distribution robustness” generally.
  • Giving the proposed method more data, tuning, compute, or external pretraining.
  • Omitting failed runs and choosing the best seed.
  • Publishing code without data identity, split IDs, configuration, or environment.
  • Reporting only trainable parameters while hiding search and pretraining resources.

Self-check

Section titled “Self-check”

Without notes, answer:

  1. What population quantity does the primary metric estimate?
  2. Which unit is independently sampled, split, and resampled?
  3. Why does calibration require a population and time horizon?
  4. When can ROC AUC remain stable while precision changes?
  5. Why is a paired interval usually more informative for two models on the same cases?
  6. Which uncertainties does a softmax vector fail to separate?
  7. What guarantee does split conformal prediction provide, and under what condition?
  8. How would you distinguish a component gain from a parameter-count gain?
  9. What files are necessary to regenerate a table without rerunning training?
  10. Which deployment change lies outside the evidence?

Exit task

Section titled “Exit task”

Take one result you intend to cite. Write its claim–estimand–design contract, redraw the complete data and selection boundary, recompute a paired uncertainty interval from saved predictions, and identify one plausible leakage or shift. Produce a compact evaluation card with metrics, calibration, subgroups, compute, failure cases, and deployment boundary, plus a reproducibility bundle another person can run from a clean environment.

Atlas-conditioned foundation segmentation and reliability fusion

A single annotated volume can generate spatial prompts and an independent registered-mask prediction. AtlasSegFM learns a voxel-wise fusion gate through support-space consistency while leaving the foundation backbone frozen. Read it as a one-shot supervision budget with substantial test-time adaptation.

Primary paper · year · evidenceCore idea and why to read itOfficial resources
Atlas is Your Perfect Context: One-Shot Customization for Generalizable Foundational Medical Image SegmentationZiyu Zhang, Yi Yu, Simeng Zhu, Ahmed Aly, Yunhe Gao, Ning Gu, and Yuan Xue2026 · ECCV · acceptedECCV 2026 Spotlight verified in the official program, session Computational, Medical & Scientific Imaging; scientific claims use conference-hosted manuscript #2028. Publisher proceedings/version of record was not verified on 2026-09-12.AtlasSegFM customizes a frozen segmentation foundation model using one annotated atlas. It registers atlas anatomy to the query, converts the warped mask into prompts, and combines atlas and foundation-model predictions with a voxel-wise reliability gate. One-shot refers to the support annotation budget, not absence of test-time optimization.ECCV Spotlight program · Conference manuscript

Reading question: What is labeled, what is optimized for each query, and can the gate be trained without using query ground truth?

Evidence boundary: Anatomical mismatch can corrupt both branches; interactive methods receiving ground-truth-derived clicks have a different information budget. CT/MRI anatomy is not WSI validation.

Resolution transfer and calibrated geophysical fields

BaguanHR creates additional fine-resolution atmospheric training fields through super-resolution, whereas Sparse2Cloud predicts dense cloud geometry from sparse active-sensor tracks and calibrates quantile intervals. Compare transferred information, geographic or temporal splits and uncertainty, not just grid density.

Primary paper · year · evidenceCore idea and why to read itOfficial resources
Pushing the Limits of High-Resolution Weather Forecasting through Data ScalingYang Zhao, Peisong Niu, Tian Zhou, Ziqing Ma, Guanlong Ma, Rong Jin, Huiling Yuan, and Liang Sun2026 · ECCV · acceptedECCV 2026 Spotlight verified in the official program, session Computational, Medical & Scientific Imaging; scientific claims use conference-hosted manuscript #7258. Publisher proceedings/version of record was not verified on 2026-09-12.BaguanHR transfers high-resolution data rather than merely transferring a coarse forecast model. Variable-specific super-resolution expands scarce 0.1-degree atmospheric training data from long 0.25-degree ERA5 records, then a high-resolution forecaster learns from synthetic-plus-real fields. Adaptive rollout weighting and replay target long-horizon error accumulation.ECCV Spotlight program · Conference manuscript · Author project (paper/program link)
Ice Cloud Geometry Retrieval with Calibrated Uncertainty from Passive Satellite ImageryAyush Prasad2026 · ECCV · acceptedECCV 2026 Spotlight verified in the official program, session Computational, Medical & Scientific Imaging; scientific claims use conference-hosted manuscript #12985. Publisher proceedings/version of record was not verified on 2026-09-12. Selected aggregate values follow Table 1. Table 2 reports a different G-Q aggregate R2 (0.725 vs 0.731); keep table-specific provenance. CQR guarantees marginal coverage only under exchangeability, not arbitrary subgroups or global domain shift.Sparse2Cloud learns dense ice-cloud geometry from narrow satellite supervision tracks and attaches conformal prediction intervals. A ConvNeXt U-Net predicts eight cloud targets and three quantiles per target, conditioned on passive VIIRS channels and ERA5 profiles. A five-model quantile ensemble plus calibration separates point accuracy from interval coverage.ECCV Spotlight program · Conference manuscript · Official implementation · Author project (paper/program link)

Reading question: Does a finer output represent measured extra information, learned prior interpolation or a calibrated uncertainty statement?

Evidence boundary: Synthetic resolution does not guarantee missing fine physics. Marginal conformal coverage assumes exchangeability and is not a universal conditional guarantee under climate or geographic shift.

Cross-magnification transfer: utility, predictability and task benefit

After XMaG and LRMIL, test whether observed teacher targets help, whether native low-power features predict them, and whether MIL benefits from the predictions. Direct versus residual regression and native-feature retention are separate controls.

Primary paper · year · evidenceCore idea and why to read itOfficial resources
What Makes High-Magnification Knowledge Transferable? A Study of Cross-Resolution Distillation in Whole-Slide ImagingZhiyuan Yang, Jiahao Cheng, and Mahdi S. Hosseini2026 · arXiv · preprintPublic arXiv v2 (2026-10-01) verified 2026-10-02. The authors state ICLR 2027 under review; no acceptance inferred. Only public source material was used. A diagnostic study of target utility, predictability and downstream benefit, not a new universally superior distillation architecture.Separate teacher usefulness, student predictability and downstream benefit. Observed 20× regional means can help slide prediction when supplied with native 5× features, yet a 5× predictor may fail to reconstruct them usefully. Lower reconstruction error does not consistently translate into better slide-level performance.Reviewed manuscript revision

Reading question: Can the proposed target beat a constant training-mean baseline and improve the final slide endpoint?

Evidence boundary: Observed 20× access is a diagnostic reference, not low-power-only deployment. Lower reconstruction MSE is not guaranteed downstream improvement; the study is an ICLR 2027 submission-stage preprint.

Uncertainty contracts: confidence, marginal coverage and set efficiency

Temperature scaling preserves class ranking but changes adaptive conformal scores. A temperature that improves confidence calibration can enlarge prediction sets while reducing empirical class-coverage disparity. Use separate objectives and report all three axes instead of calling a model simply calibrated.

Primary paper · year · evidenceCore idea and why to read itOfficial resources
On Temperature Scaling and Conformal Prediction of Deep ClassifiersLahav Dabah and Tom Tirer2025 · ICML · peer-reviewedCalibrated confidence and conformal prediction sets solve different uncertainty problems. Temperature scaling may make confidence more reliable yet enlarge adaptive prediction sets. Dabah and Tirer show this trade-off empirically and explain a non-monotonic temperature effect; they propose separate temperatures for confidence and prediction-set goals.Final proceedings PDF · Author implementation · Author manuscript

Reading question: Is the target honest confidence, a small candidate set, or reliable coverage for a specified population?

Evidence boundary: Marginal exchangeability-based coverage is not per-class, per-hospital or per-patient certainty. Temperature selection, calibration units and final test data must remain distinct.

Global semantic anchors and local patch targets

Two Global Crops Suffice separates same-image global-view alignment from local crops and iBOT refinement in controlled DINO-family experiments. CODI instead changes the patch target: weakly mix the same teacher view’s class distribution into its local patch distribution. The diagnostic study and target-side method answer related questions, but neither independently validates the other.

Primary paper · year · evidenceCore idea and why to read itOfficial resources
Two Global Crops Suffice: Locating Semantic Emergence in DINO-Style Self-Supervised LearningBasavaraj Sunagad, Artur Jesslen, and Adam Kortylewski2026 · arXiv · preprintarXiv:2609.28187v1, submitted 23 September 2026, inspected 26 October 2026. Empirical DINO-family ablations on ImageNet-1k, not a universal necessity theorem or an evaluation of DINOv3/pathology. The review retains a discrepancy between the main text’s fixed-scale depth claim and Appendix Table A3.This is a controlled diagnosis of DINO-style learning, not a new foundation model. In the tested ImageNet-1k regime, matching two geometrically different global crops from the same image already produces semantic patch correspondence. iBOT’s masked-patch objective improves that structure when combined with the global objective, while extra local crops chiefly improve classification in the DINO-only comparison. The practical lesson is to separate semantic anchoring, refinement and evaluation rather than treat a successful recipe as an indivisible block.Inspected manuscript and appendices (v1)

Reading question: Which signal creates useful spatial semantics, which refines it, and do your global and dense evaluations agree?

Evidence boundary: Both inspected sources are preprints. Crop ablations match iterations, not all computation; the depth-scale narrative has an appendix exception. CODI’s gradient statement is local/logit-level, its alpha is task-dependent, and its code was not released at inspection. Neither reports WSI or cross-magnification validation.

Complete maintained paper route

Section titled “Complete maintained paper route”

37 canonical readings · 1979–2026· route reviewed through October 2, 2026.

This is the complete structured list for this chapter, ordered chronologically from earlier foundations toward current work. The explanatory ladder above highlights the shortest conceptual route; this list also includes important branches, benchmarks, and current endpoints. It is generated from the same deduplicated catalog used by the AI background registry and My Library, so a future catalog update appears here automatically.

Show all 37 readings in maintained order

  14. #142021Communications of the ACMPeer reviewedL1–L2

    Datasheets for Datasets

    Timnit Gebru et al.

    Metadata checked

  22. #222026Findings of EMNLPAccepted · proceedings pendingL3

    MVC-Bench: Benchmarking Calibration of Medical Vision-Language Models

    Ashshak Sharifdeen, Shihab Aaqil Ahamed, Ufaq Khan, Muhammad Akhtar Munir Sujair Ibrahim, Mohamed Rafeek Mareer Ahamed, Yutong Xie, Imran Razzak, and Muhammad Haris Khan

    The arXiv record reports acceptance to Findings of EMNLP 2026. No final ACL Anthology record was available when checked on 30 August 2026, so claims were verified against arXiv v1 and the official repository.

    Official artifacts: Official implementation

    Metadata checked

  28. #282026arXivPreprintL3

    Sliding-window beats linear attention

    Alexia Jolicoeur-Martineau, Rhea Sanjay Sukthanker, Pashmina Cameron, and Emy Gervais

    arXiv v1 submitted 28 August 2026. No peer-reviewed record, code release, or model artifact was verified on 2 September 2026; it is retained for its unusually important negative baseline evidence rather than publication status.

    Metadata checked

  29. #292026arXivPreprintL3

    Puro-2B: Poor Lab's Qwen2-1.5B Trained on RTX 5090 within $5090

    Kairong Luo, Jiarui Cui, Yaorui Yin, Shengqi Chen, Yiming Yang, Linxiang Gao, Yanmohan Wang, Mingzhe Zhang, Kaiyue Wen, Kaifeng Lyu, and Wenguang Chen

    arXiv v1 submitted 27 August 2026. Catalog authors follow the current arXiv metadata; the PDF title page additionally names Chengxia Li, an unresolved source-metadata discrepancy that should be rechecked on revision. Code, data, and checkpoints were released, but no peer-reviewed record was verified.

    Official artifacts: Official training implementation · Official models and data collection

    Metadata checked

  31. #312026EMNLPAccepted · proceedings pendingL3

    IntroConformal: Conformal Factuality Guarantees for Large Vision-Language Models via Introspective Signals

    Md. Atabuzzaman, Christian Alexander, and Chris Thomas

    The arXiv v1 record submitted 1 September 2026 reports acceptance to the EMNLP 2026 main conference. No final ACL Anthology record was available when checked on 2 September 2026, so publication claims remain author-reported and scientific claims use the arXiv manuscript.

    Official artifacts: Official implementation

    Metadata checked

  32. #322026arXivPreprintL3

    The Safeguard Worked. Is the LLM System Safer?

    Pingyu Wu, Weiming Zhang, and Nenghai Yu

    arXiv v1 submitted 1 September 2026. No peer-reviewed record or first-party machine-readable evidence artifact was verified; it is retained because its 198-paper evidence audit supplies unusually important negative system-level evidence.

    Metadata checked

  33. #332026ECCVAccepted · proceedings pendingL3

    Atlas is Your Perfect Context: One-Shot Customization for Generalizable Foundational Medical Image Segmentation

    Ziyu Zhang, Yi Yu, Simeng Zhu, Ahmed Aly, Yunhe Gao, Ning Gu, and Yuan Xue

    ECCV 2026 Spotlight verified in the official program, session Computational, Medical & Scientific Imaging; scientific claims use conference-hosted manuscript #2028. Publisher proceedings/version of record was not verified on 2026-09-12.

    Official artifacts: ECCV Spotlight program · Conference manuscript

    Metadata checked

  34. #342026ECCVAccepted · proceedings pendingL3

    Pushing the Limits of High-Resolution Weather Forecasting through Data Scaling

    Yang Zhao, Peisong Niu, Tian Zhou, Ziqing Ma, Guanlong Ma, Rong Jin, Huiling Yuan, and Liang Sun

    ECCV 2026 Spotlight verified in the official program, session Computational, Medical & Scientific Imaging; scientific claims use conference-hosted manuscript #7258. Publisher proceedings/version of record was not verified on 2026-09-12.

    Official artifacts: ECCV Spotlight program · Conference manuscript · Author project (paper/program link)

    Metadata checked

  35. #352026ECCVAccepted · proceedings pendingL3

    Ice Cloud Geometry Retrieval with Calibrated Uncertainty from Passive Satellite Imagery

    Ayush Prasad

    ECCV 2026 Spotlight verified in the official program, session Computational, Medical & Scientific Imaging; scientific claims use conference-hosted manuscript #12985. Publisher proceedings/version of record was not verified on 2026-09-12. Selected aggregate values follow Table 1. Table 2 reports a different G-Q aggregate R2 (0.725 vs 0.731); keep table-specific provenance. CQR guarantees marginal coverage only under exchangeability, not arbitrary subgroups or global domain shift.

    Official artifacts: ECCV Spotlight program · Conference manuscript · Official implementation · Author project (paper/program link)

    Metadata checked

  36. #362026arXivPreprintL3

    What Makes High-Magnification Knowledge Transferable? A Study of Cross-Resolution Distillation in Whole-Slide Imaging

    Zhiyuan Yang, Jiahao Cheng, and Mahdi S. Hosseini

    Public arXiv v2 (2026-10-01) verified 2026-10-02. The authors state ICLR 2027 under review; no acceptance inferred. Only public source material was used. A diagnostic study of target utility, predictability and downstream benefit, not a new universally superior distillation architecture.

    Official artifacts: Reviewed manuscript revision

    Metadata checked

  37. #372026arXivPreprintL3

    Two Global Crops Suffice: Locating Semantic Emergence in DINO-Style Self-Supervised Learning

    Basavaraj Sunagad, Artur Jesslen, and Adam Kortylewski

    arXiv:2609.28187v1, submitted 23 September 2026, inspected 26 October 2026. Empirical DINO-family ablations on ImageNet-1k, not a universal necessity theorem or an evaluation of DINOv3/pathology. The review retains a discrepancy between the main text’s fixed-scale depth claim and Appendix Table A3.

    Official artifacts: Inspected manuscript and appendices (v1)

    Metadata checked