Skip to content
Computational Pathology Reading Roadmap

Spatial, molecular, and outcome modeling

Computational pathology increasingly asks questions beyond diagnosis: can morphology predict a mutation, pathway, gene-expression pattern, treatment-relevant biomarker, or patient outcome? Can cells and tissue regions be represented as spatial systems rather than an unordered bag? These are powerful research directions, but their labels, confounders, and claims require more care than a standard image classifier.

Latest literature discovery audit: 2 October 2026. Existing paper reviews retain their individual source-check dates.

This chapter joins four related tracks: molecular prediction from H&E, spatial transcriptomics–histology learning, graph/pathomics models, and multimodal prognosis. Read MIL and WSI learning first for slide aggregation and cell and tissue modeling first for object-level spatial features.

1. Separate the scientific targets

Section titled “1. Separate the scientific targets”
Track Input at inference Target during training/evaluation Defensible first claim
Molecular screening H&E WSI mutation, MSI, expression, pathway, protein/IHC morphology contains signal useful for a defined screening setting
Bulk expression prediction H&E WSI case-level RNA-seq selected expression/signatures are predictable in the studied cohort
Spatial expression prediction H&E aligned to spatial assay spot/bin/cell expression morphology predicts spatial measurements at a stated resolution
Histology–omics fusion H&E plus omics at inference diagnosis, response, survival modalities add complementary predictive information
Spatial/pathomics graph detected cells, regions, or patches phenotype or outcome modeled relations improve or explain a defined endpoint
Survival/prognosis H&E, optionally omics/clinical data censored time-to-event model ranks or calibrates risk in the target population

Do not collapse these into “virtual omics.” Predicting a measured assay retrospectively does not establish analytical equivalence, clinical utility, or replacement of the assay. Use language proportional to the experiment.

2. Molecular prediction from routine histology

Section titled “2. Molecular prediction from routine histology”

The central causal ambiguity is:

molecular state → tissue phenotype → pixels
        ↘ disease, site, treatment, sampling, prevalence ↗

A model can exploit a phenotype genuinely associated with the target, a correlated disease subtype, site/scanner artifacts, treatment effects, or cohort prevalence. External-site testing is necessary but not sufficient; the target assay, specimen timing, tissue composition, and clinical spectrum must also match.

Foundational reading route

Section titled “Foundational reading route”
Order Primary paper Year / status Read for
1 Classification and Mutation Prediction from Non–Small Cell Lung Cancer Histopathology Images 2018 · Nature Medicine joint histology classification and selected mutation prediction from H&E
2 Deep Learning Can Predict Microsatellite Instability Directly from Histology in Gastrointestinal Cancer 2019 · Nature Medicine weakly supervised MSI screening and multi-cohort testing
3 A Deep Learning Model to Predict RNA-Seq Expression of Tumours from Whole Slide Images (HE2RNA) 2020 · Nature Communications systematic bulk expression prediction and gene/signature-level evaluation
4 Development and Validation of a Weakly Supervised Framework to Predict Molecular Pathways and Mutations in Colorectal Cancer 2021 · Lancet Digital Health multi-target prediction, external testing, and target-specific performance
5 Pacpaint 2023 · Nature Communications regional tumour/stroma molecular subtype prediction and intratumour heterogeneity

Design requirements

Section titled “Design requirements”
  • Define the assay, platform, calling pipeline, threshold, and timing relative to the slide. A gene mutation, RNA abundance, pathway score, and IHC surrogate are different labels.
  • Use one patient in one split. Multiple slides may share one assay result.
  • State whether tumour regions are annotated, automatically selected, or discovered.
  • Prespecify the clinical operating point; AUROC alone can hide low positive predictive value in a low-prevalence population.
  • Test calibration, sensitivity, specificity, PPV/NPV, and decision consequences at deployment prevalence.
  • Compare against accessible clinicopathologic variables and a combined model.
  • Audit whether site predicts both label and image appearance.

3. Spatial transcriptomics and histology

Section titled “3. Spatial transcriptomics and histology”

Spatial assays pair morphology with gene measurements, but the alignment is imperfect and the measurement unit changes across platforms. A Visium spot can contain multiple cells; newer platforms approach cellular or subcellular resolution. Record platform, spot/bin size, tissue thickness, section relationship, registration, filtering, normalization, gene selection, and spatial smoothing.

Method progression

Section titled “Method progression”
Order Primary paper/resource Year / status Contribution to understand
1 Integrating Spatial Gene Expression and Breast Tumour Morphology via Deep Learning (ST-Net) 2020 · Nature Biomedical Engineering local patch-to-expression prediction on paired breast tissue
2 Spatial Transcriptomics Prediction from Histology Jointly through Transformer and Graph Neural Networks (Hist2ST) 2022 · Briefings in Bioinformatics joins image features, spatial context, Transformer/GNN modeling, and expression targets
3 Spatially Resolved Gene Expression Prediction via Bi-modal Contrastive Learning (BLEEP) 2023 · NeurIPS contrastive alignment of image and expression embeddings
4 HEST-1k 2024 · NeurIPS Datasets & Benchmarks standardized multi-cohort data, a library, and foundation-model benchmark tasks
5 STPath 2025 · npj Digital Medicine generative pretraining across large paired ST–histology collections
6 STFlow 2025 · ICML, peer reviewed whole-slide conditional flow matching for generating spatial transcriptomics from histology; compare with deterministic predictors under patient/spatial splits and assess distributional calibration, not only point accuracy
7 FineST 2026 · Nature Communications contrastive histology–ST fusion for finer-grained analysis
8 HESpotEx 2026 · Nature Computational Science dual-stream image/graph framework and cross-platform spatial-expression tests
9 MSGR: Gene Ontology-Guided Hierarchical Spatial Gene Expression Prediction 2026 · ACM Multimedia accepted according to arXiv; proceedings pending makes gene-set and ontology hierarchy part of the prediction architecture; author code, with patient-level and spatial-only baselines still required

HEST-1k is also listed in the WSI Data Atlas. Inspect its license, cohort provenance, and overlap with any encoder pretraining data before calling it an independent benchmark.

Spatial foundation models and virtual assays

Section titled “Spatial foundation models and virtual assays”

These papers form a distinct methodological branch: they use paired spatial assays to pretrain transferable representations, infer measurements at finer resolution, or construct virtual molecular maps. Keep predicted maps separate from measurements made by an assay.

Paper Evidence state Scientific distinction
iStar Nature Biotechnology 2024, peer reviewed super-resolution tissue architecture from histology plus ST; reconstructed values are not independently measured single-cell expression
OmiCLIP / Loki Nature Methods 2025, peer reviewed visual–omics pretraining over paired patches plus alignment, annotation, decomposition, retrieval, and expression tasks
spEMO Nature Biomedical Engineering 2026, peer reviewed combines pathology-FM and LLM representations across spatial-domain, cell-type, slide, interaction, and reporting tasks; evidence is computational benchmarking, not deployment
HEX virtual spatial proteomics Nature Medicine 2026, peer reviewed predicts 40 protein-marker maps and studies associations; predicted maps must not be described as measured assays or replacements for prospective testing
HistoCell and sCellST Nature Communications 2025 / 2026, peer reviewed cell-resolution inference dependent on nuclei detection, deconvolution, weak supervision, and limited cancer cohorts
AI-guided spatial proteomics in TNBC 2026, arXiv preprint an H&E recurrence model generates coordinate-level risk niches that guide mass-spectrometry sampling; 46 regions from two recurrence patients provide spatial molecular evidence, so discovery, replication, and outcome validation must remain distinct

Evaluation must defeat easy baselines

Section titled “Evaluation must defeat easy baselines”

For each gene or signature, compare against:

  1. training-set mean or median;
  2. organ/patient/site mean;
  3. spatial-neighbour interpolation without histology;
  4. coordinates or tissue-region labels without image features; and
  5. image features with spatial context removed.

Split by patient or cohort, never random spots. Report performance across genes, not only the best selected genes; account for multiple testing; and separate within-slide interpolation from generalization to a new patient or site. A spatially smooth output can correlate well while missing local biology.

4. Graph and spatial-context models

Section titled “4. Graph and spatial-context models”

A WSI can become several different graphs:

Graph Node Edge Main question
Cell graph nucleus/cell radius, k-nearest, contact, learned which cell neighbourhoods or interactions matter?
Region graph gland, tumour nest, compartment adjacency or distance how is tissue architecture organized?
Patch graph WSI tile grid adjacency, feature similarity, learned does explicit slide topology improve aggregation?
Heterogeneous graph multiple cell/region types typed relations which cross-scale relations support the endpoint?

Graph construction is part of the model. Report segmentation/detection provenance, node attributes, coordinate scale, edge rule, degree distribution, disconnected components, graph size, pooling, and sensitivity to the rule.

Key readings:

  1. Whole Slide Images Are 2D Point Clouds: Patch-GCN (2021 · MICCAI) — spatial patch graphs and survival modeling.
  2. SlideGraph+: Whole Slide Image Level Graphs to Predict HER2 Status (2022 · Medical Image Analysis) — WSI graph construction for a molecular endpoint.
  3. OCELOT (2023 · CVPR) — cell predictions informed by overlapping tissue context.

Compare every GNN with a parameter-matched set/MIL model and with graph edges permuted or ablated. Otherwise an apparent graph gain may come from different features, pooling, or capacity rather than spatial relations.

5. Histology–omics fusion

Section titled “5. Histology–omics fusion”

Fusion is meaningful only if both modalities are available at the intended inference time. If omics is used to train a histology encoder but is absent at inference, call it multimodal pretraining or privileged information—not a multimodal deployed model.

Primary paper Year / status Fusion lesson
Pathomic Fusion 2022 · IEEE TMI tensor-based integration of histology and genomic features
MCAT 2021 · ICCV co-attention between WSI instances and genomic embeddings
PORPOISE 2022 · Cancer Cell pan-cancer histology–genomic prognosis and multimodal explanations
MOTCat 2023 · ICCV optimal-transport alignment and memory-aware co-attention
SurvPath 2024 · CVPR pathway-grouped transcriptomic tokens and dense pathway–patch interactions
Multimodal Prototyping 2024 · ICML compact learned prototypes before multimodal fusion
THREADS 2025 · arXiv preprint molecularly driven slide-level foundation pretraining; treat claims as provisional until a version of record
PANDA: Prototype-Anchored Alignment 2026 · arXiv preprint class-prototype alignment for partially/unpaired auxiliary modalities; the TCGA lung experiment deploys WSI without RNA, so classify it as training-time privileged information rather than test-time fusion

Fusion ablations

Section titled “Fusion ablations”

Always report:

  • clinical-only, image-only, omics-only, and combined models;
  • identical patients and folds across ablations;
  • missing-modality frequency and a deployment-time policy;
  • early, late, and/or interaction fusion under a comparable budget;
  • calibration and uncertainty, not only ranking;
  • performance by site and disease subtype; and
  • whether omics preprocessing was fit inside each training fold.

For PANDA, additionally report paired and unpaired counts, complete-case selection, the missingness mechanism, prototype construction, histology-only and shuffled-pair controls, and uncertainty for survival comparisons. The manuscript promises code upon acceptance but had no public implementation at the cutoff. Prototype alignment does not establish that a histologic region expresses a transcript without spatially matched ground truth.

6. Survival and time-to-event outcomes

Section titled “6. Survival and time-to-event outcomes”

Survival labels combine an event, time origin, follow-up, censoring, competing events, and treatment era. State all of them. Overall survival, disease-specific survival, progression-free survival, and recurrence are not interchangeable.

Common objectives include Cox partial likelihood, discrete-time hazards, accelerated failure-time models, and ranking losses. Match the model to the estimand rather than choosing a loss by popularity.

Report:

  • Harrell and/or Uno concordance with confidence intervals;
  • time-dependent discrimination at clinically meaningful horizons;
  • calibration curves or error at those horizons;
  • integrated Brier score where appropriate;
  • risk groups defined on training data, not the test set;
  • number at risk and censoring distribution; and
  • added value over stage, grade, age, and other available clinical baselines.

Do not interpret a significant log-rank test on model-defined test-set groups as proof of individual risk calibration or clinical utility.

Censoring-aware conformal lower predictive bounds (2026, arXiv preprint) ask a more operational question: for a chosen error level, how can a pathology survival model issue a conservative patient-level time bound while accounting for censoring? Read the multi-cohort coverage results beside ordinary discrimination and calibration. Marginal conformal coverage is not subgroup coverage, does not repair a biased survival model, and depends on exchangeability, censoring estimation, calibration-set construction, and the exact bound definition.

7. A rigorous end-to-end protocol

Section titled “7. A rigorous end-to-end protocol”
clinical_question: screening | spatial inference | prognosis | multimodal decision
prediction_time: when inputs are available
population: eligibility, sites, dates, treatments
unit_and_hierarchy: patient > case > slide > region/spot/cell
target_measurement: assay/platform/calling or event/censoring definition
registration: section relation, coordinates, transform, QC
split: patient/site/temporal policy
image_pipeline: WSI QC, MPP, sampling, encoder release
omics_pipeline: normalization, feature selection, fit scope
spatial_graph: nodes, edges, scale, sensitivity analysis
baselines: prevalence/spatial/clinical/unimodal
metrics: discrimination + calibration + uncertainty
external_test: assay and population compatibility
claim: limited to evidence level
Section titled “8. Recommended synthesis projects”

Project A — molecular screening audit

Section titled “Project A — molecular screening audit”

Reproduce one mutation or MSI task with a frozen foundation encoder and MIL. Add a clinical baseline, image-plus-clinical model, site classifier, calibration, and external cohort. Write an error analysis by tumour content and site.

Project B — spatial-expression benchmark

Section titled “Project B — spatial-expression benchmark”

Use a HEST cohort with patient-level splits. Compare mean, spatial-neighbour, image-only, spatial-only, and image-plus-spatial models. Evaluate prespecified genes, pathways, held-out patients, and held-out cohorts.

Project C — graph necessity

Section titled “Project C — graph necessity”

Start from validated cell or patch features. Compare set pooling, coordinate-aware pooling, the proposed graph, permuted edges, and multiple edge radii. Report whether the scientific conclusion survives segmentation noise.

Project D — multimodal prognosis

Section titled “Project D — multimodal prognosis”

Build clinical-only, histology-only, omics-only, and fused models on identical folds. Add missing-modality simulation, calibration at fixed horizons, and a temporal or external test. Conclude whether fusion adds robust information—not merely whether it wins one cross-validation average.

For evidence levels and reporting standards, continue to evaluation and clinical translation.

Pathway-structured privileged supervision

MoPE uses training-only RNA to supervise WSI pathway experts through shared-memory distributions. It aligns a structured view of molecular information without requiring RNA at inference.

Primary paper · year · evidenceCore idea and why to read itOfficial resources
Pathway-Structured Privileged Distillation for Deployable Computational PathologyYongxin Guo, Hao Lu, Onur C. Koyun, Zhengjie Zhu, Muhammet F. Demir, and Metin N. Gurcan2026 · arXiv · preprintarXiv v1 (2026-06-01) checked 2026-10-02; no conference acceptance verified. Paired RNA is training-only, while WSI is used at deployment. Pathway-memory readouts are not independently validated molecular measurements.MoPE learns pathway-structured histology predictors using paired bulk RNA during training, but deploys with WSI alone. Instead of forcing RNA and tissue embeddings to be identical, it aligns their distributions over a shared memory. This is privileged-modality distillation, not magnification reduction or direct RNA reconstruction.Versioned manuscript

Reading question: Which part of the molecular target is transferable from morphology, and what independent evidence supports biological interpretation?

Evidence boundary: Paired RNA is still needed for training. Model attention is not measured pathway activation; this is neither optical cross-magnification transfer nor prospective clinical validation.

From fixed tissue graphs to evolving concept-guided fields

TMEvolve groups frozen CONCHv1.5 patch features into soft semantic regions, exchanges messages on a spatial graph, and lets regional and local states repeatedly update each other before slide prediction. Concept guidance and task supervision do not make its latent regions measured cell types or biological dynamics.

Primary paper · year · evidenceCore idea and why to read itOfficial resources
Modeling Whole-Slide Images as Dynamic Tumor Microenvironment FieldsLei Wu, Jiashuai Liu, Di Zhang, Zhangpeng Gong, Yingkang Zhan, Yi Niu, Jiusong Ge, Chunze Yang, Kai Yi, Mireia Crispin-Ortuzar, Chen Li, and Zeyu Gao2026 · NeurIPS · acceptedAccepted as a NeurIPS 2026 poster: official conference program event 149764 matches the exact title and 12 authors, checked 2026-10-08. Final proceedings were not verified; review uses arXiv v1 (2026-09-28). Official method code is available under a noncommercial research license, but does not include the WSI feature-extraction pipeline. Tissue-tile pixel size, extraction magnification/MPP, and image preprocessing remain unreported in the inspected paper and release.TMEvolve replaces a static bag of patch features with adaptive tissue regions whose states are repeatedly refined. Similar-region neighbors exchange visual information; heterogeneous boundaries carry directional visual and pathological-concept messages. The final regions are pooled for slide-level prediction. This is reaction-diffusion-inspired representation learning, not a simulation of tumor progression or a generative diffusion model.Reviewed manuscript and supplement (v1) · Author publication and acceptance statement · Official NeurIPS 2026 poster · Official method implementation (noncommercial research)

Reading question: Does adaptive region–patch interaction add value beyond the same encoder with set pooling or a fixed spatial graph?

Evidence boundary: Tile pixel size, extraction magnification and MPP are not reported in the inspected paper or method release. The 512 setting is a hidden feature width, not pixels; offline feature extraction is excluded from the reported method cost.

Spatial evidence: separate coherence, molecular agreement, and anatomy

SpaPath-Bench compares frozen image features through a controlled spatial learner and clustering readout on paired histology–ST sections. Its three reference regimes expose different strengths: a smooth partition, a transcriptomic proxy, and an expert anatomical boundary are not interchangeable targets.

Primary paper · year · evidenceCore idea and why to read itOfficial resources
Benchmarking Pathology Foundation Models for Spatial Domain UnderstandingBokai Zhao, Yiyang Zhang, Yuanchi Zhu, Hanqing Chao, Long Bai, Tai Ma, Minfeng Xu, Ming Song, and Tianzi Jiang2026 · MICCAI · peer-reviewedMICCAI 2026 chapter, LNCS 16895, pages 35–45, first online 25 September 2026; the volume imprint says 2027. Review uses the inspected publisher PDF. The 42 slides comprise 12 expert-annotated DLPFC slides and 30 slides assessed using transcriptomic clustering proxies. The linked public repository is a prepared-results dashboard, not a verified end-to-end benchmark implementation.SpaPath-Bench asks whether frozen pathology features support meaningful tissue partitions after a controlled spatial readout. It compares 19 encoders on 42 paired H&E–spatial-transcriptomics slides, separating spatial coherence, molecular-partition agreement and expert-label agreement. The evaluated object is an encoder-plus-readout system, not innate global spatial reasoning by a local tile encoder.Publisher full chapter · Official MICCAI open-access record · Official accepted manuscript · Author results dashboard · Dashboard repository and documented limits

Reading question: Which spatial relationship is preserved, and how much is supplied by the readout rather than the encoder?

Evidence boundary: Only 12 DLPFC slides carry expert labels; the other 30 use molecular clustering proxies. Repeated configurations are not independent specimens, and single-slide visual examples are not cohort means.

Complete maintained paper route

Section titled “Complete maintained paper route”

47 canonical readings · 2018–2026· route reviewed through October 2, 2026.

This is the complete structured list for this chapter, ordered chronologically from earlier foundations toward current work. The explanatory ladder above highlights the shortest conceptual route; this list also includes important branches, benchmarks, and current endpoints. It is generated from the same deduplicated catalog used by the AI background registry and My Library, so a future catalog update appears here automatically.

Show all 47 readings in maintained order

  44. #442026Engineering Applications of Artificial IntelligencePeer reviewedL3

    Cross-Stain Knowledge Distillation for Low-Cost Lung Cancer Programmed Death Ligand-1 Assessment With Multi-Granularity Multiple Instance Learning

    Yi Shi, Chong Ge, Fang Zhao, Anli Zhang, Ao Li, Haibo Wu, and Minghui Wang

    Peer-reviewed journal article, volume 167, article 113762. IHC is privileged teacher information during training; deployment uses the H&E student, but its macro- and micro-granularity branches remain. No official code or model release was verified on 2026-09-02.

    Metadata checked

  46. #462026NeurIPSAccepted · proceedings pendingL3

    Modeling Whole-Slide Images as Dynamic Tumor Microenvironment Fields

    Lei Wu, Jiashuai Liu, Di Zhang, Zhangpeng Gong, Yingkang Zhan, Yi Niu, Jiusong Ge, Chunze Yang, Kai Yi, Mireia Crispin-Ortuzar, Chen Li, and Zeyu Gao

    Accepted as a NeurIPS 2026 poster: official conference program event 149764 matches the exact title and 12 authors, checked 2026-10-08. Final proceedings were not verified; review uses arXiv v1 (2026-09-28). Official method code is available under a noncommercial research license, but does not include the WSI feature-extraction pipeline. Tissue-tile pixel size, extraction magnification/MPP, and image preprocessing remain unreported in the inspected paper and release.

    Official artifacts: Reviewed manuscript and supplement (v1) · Author publication and acceptance statement · Official NeurIPS 2026 poster · Official method implementation (noncommercial research)

    Metadata checked

  47. #472026MICCAIPeer reviewedL3

    Benchmarking Pathology Foundation Models for Spatial Domain Understanding

    Bokai Zhao, Yiyang Zhang, Yuanchi Zhu, Hanqing Chao, Long Bai, Tai Ma, Minfeng Xu, Ming Song, and Tianzi Jiang

    MICCAI 2026 chapter, LNCS 16895, pages 35–45, first online 25 September 2026; the volume imprint says 2027. Review uses the inspected publisher PDF. The 42 slides comprise 12 expert-annotated DLPFC slides and 30 slides assessed using transcriptomic clustering proxies. The linked public repository is a prepared-results dashboard, not a verified end-to-end benchmark implementation.

    Official artifacts: Publisher full chapter · Official MICCAI open-access record · Official accepted manuscript · Author results dashboard · Dashboard repository and documented limits

    Metadata checked