Generative modelling and diffusion
GAN, diffusion, latent-space, and conditioning objectives define the sampler.
Visual realism is not diagnostic validity. This chapter connects generative objectives to morphology, topology, multiscale coherence, privacy, and real-case utility.
19 maintained papers · literature reviewed through 2 October 2026
Histopathology generation ranges from patches and cell layouts to gigapixel slides, virtual stains, and cross-modal molecular synthesis. Visual realism is insufficient: morphology, topology, stain, multiscale coherence, patient privacy, pathologist validity, and downstream utility require separate controls against real held-out tissue.
GAN, diffusion, latent-space, and conditioning objectives define the sampler.
Patch, slide, specimen, and patient scales cannot be interchanged.
2014–2022
GAN and FID establish adversarial generation and feature-distribution evaluation; Pix2Pix and CycleGAN separate paired from unpaired translation; DDPM and latent diffusion add denoising and compressed generation.
| Order | Primary paper · authors · year · venue/status | Core idea and why to read it | Official resources |
|---|---|---|---|
| 01.01 | Generative Adversarial Nets | A generative adversarial network trains two differentiable models in a game. The generator maps prior noise to synthetic samples, implicitly defining a model distribution without an explicit density. The discriminator estimates whether an input came from real data rather than the generator. Discriminator gradients tell the generator how to move generated samples toward regions judged real; generation after training is one forward pass from noise, with no Markov chain or per-sample inference. | — |
| 01.02 | Image-to-Image Translation with Conditional Adversarial Networks | Pix2pix treats image-to-image translation as one paired conditional adversarial problem. A U-Net generator maps source image x to output G(x), preserving spatial detail through encoder-decoder skip connections. A PatchGAN discriminator judges overlapping local patches of the source-output pair as real or fake. Conditional adversarial loss encourages realistic local structure, while an L1 reconstruction term anchors the output to its aligned target and discourages arbitrary hallucination. | Official implementation |
| 01.03 | Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks | CycleGAN learns translation from unpaired image collections X and Y. Generator G maps X to Y and discriminator DY enforces target-domain appearance; generator F and discriminator DX learn the reverse. Cycle consistency requires F(G(x)) to reconstruct x and G(F(y)) to reconstruct y, constraining otherwise underdetermined adversarial mappings. Optional identity loss discourages unnecessary color changes when an image is already in the target domain. | Official implementation |
| 01.04 | GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium | This paper contributes two distinct tools: TTUR gives discriminator and generator separate learning-rate time scales so the faster subsystem can track a quasi-static slower one, while FID compares real and generated Inception features by the Fréchet distance between fitted Gaussians. TTUR is a training rule; FID is an evaluation estimator, and neither logically requires the other. | — |
| 01.05 | Denoising Diffusion Probabilistic Models | DDPM fixes a Markov chain that gradually destroys data with small Gaussian-noise steps and learns Gaussian reverse transitions that denoise one step at a time. Its practical parameterization trains a time-conditioned U-Net to predict the exact noise used to create a randomly selected corrupted state, connecting variational diffusion training to multi-scale denoising score matching. | — |
| 01.06 | High-Resolution Image Synthesis with Latent Diffusion Models | Latent Diffusion moves the expensive denoising process from RGB pixels into a mildly compressed two-dimensional autoencoder latent, then decodes the final latent once. A reusable first stage removes perceptually less important detail; a U-Net learns the remaining latent distribution; cross-attention maps text, layouts, classes, or other tokens into intermediate denoising features. | — |
Work: Reproduce paired and unpaired patch translation while freezing patient split, registration, preprocessing, and feature metric.
Exit evidence: Separate objective, pairing assumption, sampler, conditioning, and evaluation representation.
2023–2024
A morphology-weighted genotype-conditional DDPM establishes the patch baseline; DiffInfinite denoises overlapping random patches on a shared large canvas; PathLDM adapts latent diffusion to report-conditioned histology; coarse-to-fine diffusion reaches gigapixel scale.
| Order | Primary paper · authors · year · venue/status | Core idea and why to read it | Official resources |
|---|---|---|---|
| 02.01 | A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Images | This paper adapts a denoising diffusion probabilistic model to synthesize glioma histopathology conditioned on molecular genotype. Its morphology-focused weighting gives different denoising times unequal influence so learning emphasizes perceptually meaningful tissue structure rather than treating every noise level identically. The generated 128 by 128 patches are compared with a modified ProGAN by natural-image distribution metrics, improved precision/recall, and a limited pathologist assessment. | Author preprint |
| 02.02 | DiffInfinite: Large Mask-Image Synthesis via Parallel Random Patch Diffusion in Histopathology | DiffInfinite synthesizes pathology images larger than a conventional diffusion model’s training crop by denoising many overlapping random patches on one shared image lattice. A hierarchical pipeline first generates a large semantic mask and then conditions tissue appearance on that mask. At every diffusion timestep, newly randomized patch positions cover the canvas and their predictions are assembled before the next step, distributing seam corrections across time and allowing patches to run in parallel. | Author preprint · Official code |
| 02.03 | PathLDM: Text Conditioned Latent Diffusion Model for Histopathology | PathLDM is a text-conditioned latent diffusion model specialized for breast histopathology. It summarizes each TCGA-BRCA slide report, combines that text with patch-level tumor and tumor-infiltrating-lymphocyte classes, encodes the condition with pathology-trained PLIP, and denoises a less aggressively compressed latent representation. Cyclic positional encoding extends the text path beyond the usual CLIP token window. The central result is an ablated system whose FID falls to 7.64. | Author preprint · Official code |
| 02.04 | Diffusion-Based Generation of Histopathological Whole Slide Images at a Gigapixel Scale | This work scales a single patch diffusion model to synthetic whole-slide images as large as 65,536×65,536 pixels. It first samples a low-resolution tissue overview, then repeatedly doubles resolution. At each stage, a relaxed downsampling constraint keeps new patches compatible with the previous coarse image while allowing fine morphology to appear; shifting the patch grid after every diffusion step transfers information across temporary boundaries and suppresses seams. | arXiv |
Work: Reconstruct each sampler and separate patch fidelity, distribution coverage, text adherence, mask/layout validity, and slide coherence.
Exit evidence: State which module sees local pixels, global layout, molecular class, or report text—and which claimed scale is actually evaluated.
2019–2026
Autofluorescence-to-histology translation establishes label-free virtual staining; F2FLDM translates unpaired frozen sections toward FFPE with pathology embeddings and sparse-change regularization; whole-slide PARS multi-staining expands the scale and stain set.
| Order | Primary paper · authors · year · venue/status | Core idea and why to read it | Official resources |
|---|---|---|---|
| 03.01 | Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning | The method images an unstained FFPE tissue section with ordinary wide-field near-UV autofluorescence, then uses a learned generator to synthesize the bright-field appearance of a chosen histochemical stain. Training pairs come from the same section before and after chemical staining, aligned through coarse matching, global feature registration, and local elastic registration. A U-Net-like generator is trained with pixel, total-variation, and adversarial terms; a separate network is trained for each tissue-stain combination. | — |
| 03.02 | F2FLDM: Latent Diffusion Models with Histopathology Pre-Trained Embeddings for Unpaired Frozen Section to FFPE Translation | F2FLDM translates frozen-section histology toward formalin-fixed paraffin-embedded appearance without spatially paired training images. It represents source and target domains with pathology-pretrained embeddings, learns a frozen-to-FFPE embedding translator, inverts each source image into latent-diffusion noise, and guides reconstruction toward an interpolated target embedding. A sparsity-style L0 penalty on the difference between conditional noise predictions limits unnecessary changes intended to reduce hallucinated tissue. | Author preprint · Official project page |
| 03.03 | Label-free whole slide virtual multi-staining using dual-excitation photon absorption remote sensing microscopyAuthor correction published 10 April 2026; the version-of-record page is the canonical updated article. | The system combines 266 nm and 355 nm photon absorption remote sensing (PARS) microscopy to obtain complementary label-free radiative and non-radiative tissue contrasts. Registered chemical WSIs supervise separate RegGAN models for H&E, Masson's trichrome, PAS, and Jones methenamine silver. RegGAN adds a learnable registration network to paired image translation so residual misalignment is corrected during training. Patch-wise overlapping inference and blending reconstruct virtual whole slides, and the same PARS input can feed several independently trained stain models. | Public held-out WSI pairs (BioImage Archive) |
Work: Audit pairing or inversion, registration, identity preservation, patient separation, stain fidelity, hallucination, diagnostic endpoints, and independent-reader evidence.
Exit evidence: Show which source morphology is preserved at field, slide, specimen, and patient levels under external acquisition shift.
2025–2026
TopoCellGen controls cell topology; PathDiff uses unpaired text and masks; semantic/visual-crop diffusion joins layout with exemplar appearance; UniPath adds diagnostic semantic tokens and retrieved real-image prototypes.
| Order | Primary paper · authors · year · venue/status | Core idea and why to read it | Official resources |
|---|---|---|---|
| 04.01 | TopoCellGen: Generating Histopathology Cell Topology with a Diffusion Model | TopoCellGen generates a multi-class cell layout before synthesizing histology pixels. A count-conditioned diffusion model denoises cell maps while losses constrain total class counts, within-class topology, and between-class organization through persistent homology. A second guided diffusion model renders the layout as an H&E patch. The separation makes cell composition and spatial topology explicit controls instead of hoping they emerge from an image generator. | Official code |
| 04.02 | PathDiff: Histopathology Image Synthesis with Unpaired Text and Mask Conditions | PathDiff learns one histopathology diffusion model that can be conditioned by free text, a multiclass nuclei mask, or both, even though no training image has both annotations. A text-to-image latent diffusion backbone is trained on PathCap captions; a copied ControlNet-style branch is then trained on separate mask–image datasets while the backbone remains frozen. Empty or invalid condition markers let alternating batches share one generator. | Official code |
| 04.03 | Semantic and Visual Crop-Guided Diffusion Models for Heterogeneous Tissue Synthesis in Histopathology | This method conditions a histopathology diffusion model on two complementary signals: a semantic tissue map specifies where tissue classes should appear, and small visual crops specify how each class should look in the requested specimen style. For unannotated TCGA slides, foundation-model features are clustered into pseudo tissue types to construct conditions. The central hypothesis is that explicit spatial layout plus exemplar appearance better captures heterogeneous tissue than class labels or semantic maps alone. | Author preprint |
| 04.04 | Beyond Pixel Simulation: Pathology Image Generation via Diagnostic Semantic Tokens and Prototype Control | UniPath turns a frozen pathology understanding model into a controller for image generation. Sixty-four learnable queries extract Diagnostic Semantic Tokens from a frozen Patho-R1-7B backbone; a raw-text stream preserves literal wording; and a prototype stream retrieves morphology from an 8,000-image real-sample bank. Their tokens jointly condition a 0.6B Diffusion Transformer trained with flow matching in a frozen VAE latent space. The contribution is therefore multi-stream diagnostic and morphological control, not a new foundation encoder or a claim that distributional realism alone proves diagnostic validity. | Author preprint · Official code and data |
Work: Test cell graphs, map adherence, crop/prototype provenance, text visibility, rare morphology, patient leakage, and held-out utility separately.
Exit evidence: Every conditioning source has an auditable biological unit, split, retrieval policy, and counterfactual adherence test.
2025–2026
STFlow uses whole-slide flow matching to generate spatial transcriptomics from histology; MuPD uses a diffusion transformer and decoupled cross-modal attention across histology, RNA, and clinical text for synthesis and virtual staining.
| Order | Primary paper · authors · year · venue/status | Core idea and why to read it | Official resources |
|---|---|---|---|
| 05.01 | Scalable Generation of Spatial Transcriptomics from Histology Images via Whole-Slide Flow Matching | STFlow reframes histology-to-spatial-transcriptomics prediction from independent spot regression into iterative whole-slide generation. It initializes every spot's gene vector from a gene-expression prior and jointly refines the slide through flow matching. A local frame-averaged Transformer conditions each update on pathology-foundation-model image features, neighboring coordinates, current gene states, and time while remaining invariant to planar translation, rotation, and reflection. | Official code |
| 05.02 | A Generative Foundation Model for Multimodal Histopathology | MUPAD is a pathology diffusion transformer conditioned by three interchangeable biomedical views: an H&E image, free text, or transcriptomic pathway scores. It pre-trains on roughly 100 million 512-pixel H&E patches across 34 organs and aligns generated images to a Virchow2 pathology-feature teacher. Separate cross-attention routes keep modality-specific signals from collapsing, enabling image translation, text synthesis, RNA-conditioned tissue generation, data augmentation, and virtual multiplex imaging. | — |
Work: Validate each source–target direction under missing modalities, organ shift, patient separation, and biological markers.
Exit evidence: Show biological fidelity and downstream utility beyond a generic image metric.
Are source and target registered, inverted, or only case-linked, and which structures remain valid beyond visual similarity?
Image-to-Image Translation with Conditional Adversarial Networks · Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks · Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning · F2FLDM: Latent Diffusion Models with Histopathology Pre-Trained Embeddings for Unpaired Frozen Section to FFPE Translation · Label-free whole slide virtual multi-staining using dual-excitation photon absorption remote sensing microscopy
How are local detail, overlapping windows, and global tissue organization made coherent?
A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Images · DiffInfinite: Large Mask-Image Synthesis via Parallel Random Patch Diffusion in Histopathology · Diffusion-Based Generation of Histopathological Whole Slide Images at a Gigapixel Scale
Do generated morphology, cell relations, semantic regions, and exemplar appearance obey the requested control?
TopoCellGen: Generating Histopathology Cell Topology with a Diffusion Model · PathDiff: Histopathology Image Synthesis with Unpaired Text and Mask Conditions · Semantic and Visual Crop-Guided Diffusion Models for Heterogeneous Tissue Synthesis in Histopathology
Which requested semantics are visible in the patch, and can retrieval or report context leak patient or diagnosis information?
PathLDM: Text Conditioned Latent Diffusion Model for Histopathology · PathDiff: Histopathology Image Synthesis with Unpaired Text and Mask Conditions · Beyond Pixel Simulation: Pathology Image Generation via Diagnostic Semantic Tokens and Prototype Control
Which biological signals are inferred, hallucinated, or preserved across modalities?
Scalable Generation of Spatial Transcriptomics from Histology Images via Whole-Slide Flow Matching · A Generative Foundation Model for Multimodal Histopathology
| Family or stage | Scientific distinction | Main risk | Anchors |
|---|---|---|---|
| Paired / unpaired translation | Uses aligned examples, distribution constraints, source inversion, or feature guidance to predict another image domain. | Misregistration or identity loss can look plausible while changing diagnosis. | 2017 · 2017 · 2019 · 2025 |
| Visual distribution and coverage | Compares generated and real appearance, fidelity, and coverage across feature spaces or blinded review. | Generic features and FID do not establish diagnostic validity or privacy. | 2017 · 2023 · 2023 · 2024 |
| Morphology / topology / control adherence | Tests cell types, spatial relations, semantic maps, prototype/crop controls, stain markers, and multiscale tissue structure. | A correct marginal distribution can have impossible joint biology or copy conditioning tissue. | 2025 · 2025 · 2025 · 2026 |
| Downstream / clinical utility | Tests augmentation, translation, virtual assay, reader, or model benefit on real held-out cases. | Synthetic leakage, case overlap, or an FFPE-trained evaluator can inflate utility. | 2025 · 2025 · 2025 · 2026 · 2025 · 2026 |
Image-to-Image Translation with Conditional Adversarial NetworksUnpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks
Paired conditional translation is generalized to unpaired domains through adversarial and cycle-consistency constraints.
Virtual histological staining of unlabelled tissue-autofluorescence images via deep learningLabel-free whole slide virtual multi-staining using dual-excitation photon absorption remote sensing microscopy
Label-free field-level virtual staining expands to registered whole-slide multi-stain generation, making slide-scale coherence and patient splitting explicit.
Denoising Diffusion Probabilistic ModelsDiffusion-Based Generation of Histopathological Whole Slide Images at a Gigapixel Scale
Patch-scale denoising is reorganized into a coarse-to-fine sampler for gigapixel tissue.
Denoising Diffusion Probabilistic ModelsA Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Images
Generic noise prediction is specialized with genotype conditioning and morphology-prioritized timestep weighting for glioma patches.
A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology ImagesDiffInfinite: Large Mask-Image Synthesis via Parallel Random Patch Diffusion in Histopathology
Patch diffusion expands into mask-first, shared-canvas random-patch denoising for large tissue images.
High-Resolution Image Synthesis with Latent Diffusion ModelsPathLDM: Text Conditioned Latent Diffusion Model for Histopathology
Compressed diffusion is adapted with pathology report summaries, cyclic text positions, PLIP conditioning, and a lower-compression VAE.
Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial NetworksF2FLDM: Latent Diffusion Models with Histopathology Pre-Trained Embeddings for Unpaired Frozen Section to FFPE Translation
Unpaired distribution translation is reframed through source inversion, pathology embedding translation, and sparse conditional changes for frozen-to-FFPE conversion.
DiffInfinite: Large Mask-Image Synthesis via Parallel Random Patch Diffusion in HistopathologySemantic and Visual Crop-Guided Diffusion Models for Heterogeneous Tissue Synthesis in Histopathology
Semantic mask conditioning gains tissue-specific visual exemplars and a pseudo-map route for unannotated archives.
PathLDM: Text Conditioned Latent Diffusion Model for HistopathologyBeyond Pixel Simulation: Pathology Image Generation via Diagnostic Semantic Tokens and Prototype Control
Report-conditioned patch generation broadens into diagnostic semantic tokens, raw text, retrieved prototypes, and flow-matched DiT generation.
Diffusion-Based Generation of Histopathological Whole Slide Images at a Gigapixel ScaleScalable Generation of Spatial Transcriptomics from Histology Images via Whole-Slide Flow Matching
Gigapixel image synthesis broadens into coordinate-preserving whole-slide generation of a spatial molecular modality.
PathDiff: Histopathology Image Synthesis with Unpaired Text and Mask ConditionsA Generative Foundation Model for Multimodal Histopathology
Text and mask control broadens into a shared generative model across image, molecular, and clinical modalities.
19 canonical readings · 2014–2026· route reviewed through October 2, 2026.
This is the complete structured list for this chapter, ordered chronologically from earlier foundations toward current work. The explanatory ladder above highlights the shortest conceptual route; this list also includes important branches, benchmarks, and current endpoints. It is generated from the same deduplicated catalog used by the AI background registry and My Library, so a future catalog update appears here automatically.
Ian Goodfellow et al.
Metadata checked
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros
Official artifacts: Official implementation
Metadata checked
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros
Official artifacts: Official implementation
Metadata checked
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter
Metadata checked
Yair Rivenson, Hongda Wang, Zhensong Wei, Kevin de Haan, Yibo Zhang, Yichen Wu, Harun Günaydın, Jonathan E. Zuckerman, Thomas Chong, Anthony E. Sisk, Lindsey M. Westbrook, W. Dean Wallace, and Aydogan Ozcan
Metadata checked
Jonathan Ho, Ajay Jain, and Pieter Abbeel
Metadata checked
Robin Rombach et al.
Metadata checked
Puria Azadi Moghadam, Sanne Van Dalen, Karina C. Martin, Jochen Lennerz, Stephen Yip, Hossein Farahani, and Ali Bashashati
Official artifacts: Author preprint
Metadata checked
Marco Aversa, Gabriel Nobis, Miriam Hägele, Kai Standvoss, Mihaela Chirica, Roderick Murray-Smith, Ahmed M. Alaa, Lukas Ruff, Daniela Ivanova, Wojciech Samek, Frederick Klauschen, Bruno Sanguinetti, and Luis Oala
Official artifacts: Author preprint · Official code
Metadata checked
Robert Harb, Thomas Pock, and Heimo Müller
Official artifacts: arXiv
Metadata checked
Srikar Yellapragada, Alexandros Graikos, Prateek Prasanna, Tahsin Kurc, Joel Saltz, and Dimitris Samaras
Official artifacts: Author preprint · Official code
Metadata checked
Meilong Xu, Saumya Gupta, Xiaoling Hu, Chen Li, Shahira Abousamra, Dimitris Samaras, Prateek Prasanna, and Chao Chen
Official artifacts: Official code
Metadata checked
Mahesh Bhosale, Abdul Wasi, Yuanhao Zhai, Yunjie Tian, Samuel Border, Nan Xi, Pinaki Sarder, Junsong Yuan, David Doermann, and Xuan Gong
Official artifacts: Official code
Metadata checked
Tinglin Huang, Tianyu Liu, Mehrtash Babadi, Wengong Jin, and Rex Ying
Official artifacts: Official code
Metadata checked
Man M. Ho, Shikha Dubey, Yosep Chong, Beatrice Knudsen, and Tolga Tasdizen
Official artifacts: Author preprint · Official project page
Metadata checked
Saghir Alfasly, Wataru Uegami, Md Enamul Hoq, Ghazal Alabtah, and Hamid Tizhoosh
Official artifacts: Author preprint
Metadata checked
Jinxi Xiang, Mingjie Li, Siyu Hou, Yijiang Chen, Xiangde Luo, Yuanfeng Ji, Xiang Zhou, Ehsan Adeli, Akshay Chaudhari, Curtis P. Langlotz, Kilian M. Pohl, and Ruijiang Li
Metadata checked
James E. D. Tweel, Benjamin R. Ecclestone, James A. Tummon Simmons, and Parsin Haji Reza
Author correction published 10 April 2026; the version-of-record page is the canonical updated article.
Official artifacts: Public held-out WSI pairs (BioImage Archive)
Metadata checked
Minghao Han, Yichen Liu, Yizhou Liu, Zizhi Chen, Jingqun Tang, Xuecheng Wu, Dingkang Yang, and Lihua Zhang
Official artifacts: Author preprint · Official code and data
Metadata checked