Generative and representation learning

Representation learning asks which information should be easy to recover from an embedding. Generative modelling asks how to assign probability, sample, or transport noise into data. The areas overlap—generative objectives can learn useful features—but their evaluation targets are not interchangeable.

Latest literature discovery audit: 26 October 2026. This pass added paper-scoped checks for CODI and Two Global Crops Suffice; it did not reverify every existing paper. Older reviews retain their individual source-check dates. The previous whole-route review boundary remains 2 October 2026.

Individual ICLR oral-paper studies retain their 19 September 2026 source-check dates; the later discovery audit does not re-date that evidence.

ECCV 2026 conference additions retain their 2026-09-12 metadata-check date; the later discovery pass does not re-date that conference evidence.

Prerequisites

Complete likelihood, KL divergence, expectation, and optimization in mathematics, probability, and statistics, plus backpropagation, normalization, and attention in deep learning. You should be able to build a leakage-resistant downstream evaluation using classical machine learning.

Learning objectives

After this chapter, you should be able to:

state which invariances, equivariances, or predictive relationships create a representation-learning signal;
distinguish frozen transfer, linear probing, partial adaptation, full fine-tuning, and parameter-efficient adaptation;
compare contrastive, non-contrastive, distillation, masked-prediction, domain, meta-learning, and continual objectives;
derive the autoregressive likelihood, VAE evidence lower bound, GAN game, change-of-variables likelihood, diffusion denoising target, and flow-matching target;
choose evaluation that separates density, fidelity, coverage, conditional correctness, memorization, and downstream utility;
identify shortcuts, collapse, leakage, prior mismatch, mode dropping, and out-of-support generation;
design an equal-data, equal-budget representation or generator comparison.

What makes a representation useful?

Let an encoder produce z = fθ(x). “Good representation” is incomplete until it names:

the downstream information that should be retained;
nuisance variation that should be ignored;
transformations under which the representation should be invariant or equivariant;
the label and sample regime available for adaptation;
the target population or domain;
the readout and evaluation budget.

An invariant representation maps transformed inputs to similar embeddings. An equivariant representation changes predictably with the transformation. Cropping, colour change, rotation, sentence reordering, and masking are not universally label-preserving; the data domain determines whether an augmentation creates a valid positive pair or destroys target information.

Supervised transfer and metric learning

Supervised pretraining transfers features learned from a labelled source task. Earlier layers often encode more general patterns while later layers specialize, but transferability depends on source/target similarity, scale, architecture, and adaptation. Negative transfer is possible.

Metric learning shapes distances:

pairwise contrastive losses pull matched pairs together and separate mismatches;
triplet losses compare anchor–positive and anchor–negative distances with a margin;
proxy and classification-based metric losses replace expensive mining with learned representatives.

Batch construction and hard-negative mining define much of the training signal. False negatives may be semantically related; overly hard negatives may be annotation errors or shortcuts.

Knowledge distillation trains a student from teacher logits, features, relations, or generated targets. Temperature changes the relative information in class probabilities. A smaller student can match a teacher in one regime without inheriting its robustness, calibration, or fairness.

Self-supervised learning

Self-supervision constructs targets from unlabelled inputs. It does not mean the training signal is assumption-free.

Contrastive learning

InfoNCE-style objectives identify paired views among other examples. For a positive pair (i,j), a common term is

-log exp(sim(z_i,z_j)/τ) / Σ_k exp(sim(z_i,z_k)/τ).

Augmentations define positives, the batch or queue defines negatives, similarity and temperature shape geometry, and the projection head separates pretext geometry from the retained encoder. SimCLR demonstrates the importance of augmentation, projection, and large contrastive batches. Momentum Contrast uses a momentum encoder and queue.

When labels are available, supervised contrastive learning treats other examples of the same class as positives. Its comparison with cross-entropy must account for extra views, batch composition, the projection head, training cost, and whether the downstream classifier is trained jointly or afterward.

Non-contrastive and distillation objectives

BYOL predicts a slowly updated target encoder without explicit negatives. Stop-gradient, target dynamics, predictor asymmetry, normalization, and optimization jointly prevent collapse in practice. DINO applies self-distillation with centring and sharpening to Vision Transformers and reveals emergent spatial organization. These methods do not show that negatives are never useful; they establish different sufficient training systems.

Global semantic anchors and local patch targets

Image-level and patch-level representation quality are different questions. In Two Global Crops Suffice (2026), controlled DINO-family experiments separate global-view alignment, extra local crops and iBOT patch prediction. Two same-image global crops retain useful correspondence; local crops improve classification more strongly in the DINO-only comparison, while iBOT substantially improves correspondence when combined with the global objective. These are recipe-specific empirical findings, not a theorem that local crops or patch supervision are universally unnecessary. Equal iteration counts do not make every crop schedule equal in computation.

CODI (2026) asks a complementary question: can local patch targets receive global semantics more directly? It weakly mixes the same teacher view’s class-token distribution into its patch distribution, using a shared prototype space. The ordinary DINO class loss remains. A large mixing ratio can hurt dense predictions; its local/logit-level gradient argument does not guarantee uniform gradients throughout the backbone.

Read both as preprint evidence and compare global classification with dense transfer under explicit data and compute controls. The crop study’s appendix reports a depth-error optimum at scale 0.32, contradicting its broad main-text claim that every fixed-scale dense-task optimum is at 0.60 or above. CODI’s author repository was still a placeholder when checked on 26 October 2026. Neither study establishes a WSI or cross-magnification result.

Redundancy reduction and variance constraints

Barlow Twins drives the cross-correlation of paired embeddings toward the identity. VICReg separately encourages view invariance, per-dimension variance, and low covariance. These objectives make anti-collapse terms explicit, but hyperparameters still set the geometry.

Masked and predictive modelling

Masked autoencoders reconstruct missing image content from visible patches using an asymmetric encoder/decoder. In language, masked-token and next-token objectives predict discrete content. Other methods predict latent targets rather than raw pixels. Mask unit, ratio, decoder capacity, tokenizer/target representation, and loss determine what is learned. Reconstruction quality alone does not establish semantic utility.

Joint-embedding predictive architectures predict representations of hidden regions from visible context rather than reconstructing every pixel. This can redirect capacity from low-level detail to more abstract predictable structure, but the target encoder, mask sampler, predictor, and anti-collapse mechanism still define the learned signal.

The specialized self-supervised pathology route adds stain, tissue, magnification, tile, slide, and patient constraints.

Semi-supervised, weak, domain, and few-shot learning

Semi-supervised learning

Pseudo-labelling turns confident predictions into targets. Consistency methods require predictions to agree under perturbations. FixMatch combines weak-augmentation pseudo-labels with strong-augmentation training. Thresholding changes class coverage, and unlabelled data can hurt when its population or label space differs.

Weak supervision

Weak targets may be noisy, incomplete, heuristic, coarse, or bag-level. Multiple labelling functions can be combined only under assumptions about their errors and dependence. In multiple-instance learning, a bag target supervises a set of instances; instance interpretations are not directly observed and require separate validation.

Domain adaptation and generalization

Domain-adversarial training encourages features that support the task but confuse a domain discriminator. Alignment can fail if label prevalence, conditional mechanisms, or label spaces differ. Domain generalization trains without target-domain access; test-time adaptation uses target observations under its own leakage and stability constraints.

Always name the shift: covariate, label, concept, acquisition, site, temporal, or selection shift. Marginal feature alignment is not proof of invariant causal structure.

Meta-learning and few-shot adaptation

Metric-based meta-learning compares a query with support examples or prototypes. Prototypical Networks represents each episode class by the mean of its embedded support set, making the metric, embedding, episode sampling, and class prototype explicit. Gradient-based MAML instead learns parameters that adapt in a few steps. Results depend on the task distribution, episode construction, backbone, transductive access, and adaptation budget. A “five-shot” result is not comparable when one method also uses external pretraining or test-set statistics.

Continual learning and plasticity

Continual learning studies a stream in which tasks or distributions change. Methods regularize important parameters, rehearse stored or generated examples, isolate parameters, or dynamically expand. Evaluate:

average performance across encountered tasks;
forgetting relative to earlier checkpoints;
forward and backward transfer;
memory, compute, and task-boundary access;
performance under gradual drift rather than only clean task switches.

Elastic Weight Consolidation penalizes movement of parameters estimated to be important for earlier tasks. Its Fisher approximation, task boundaries, storage, output-head protocol, and sequential hyperparameter access are part of the result; parameter stability can also block useful adaptation.

The 2024 loss-of-plasticity study shows that standard deep networks can lose the ability to fit new data during prolonged non-stationary training and investigates maintaining diverse, unsaturated units. This is a mechanism to test, not evidence that one continual-learning recipe covers every stream.

Parameter-efficient adaptation

Full fine-tuning updates every parameter. Alternatives include:

linear probes or frozen-feature models;
bias-only or normalization-only tuning;
adapters inserted between frozen layers;
prompt or prefix parameters;
low-rank updates such as LoRA;
quantized-backbone adaptation;
selective unfreezing and low-learning-rate layer groups.

Count trainable parameters, stored base models, optimizer states, activation memory, training compute, inference latency, and per-task artifacts. “Parameter efficient” does not necessarily mean faster or less memory during training. Compare under the same data, base checkpoint, search budget, and deployment constraint.

Evaluating representations

No single protocol identifies intrinsic quality.

Protocol	What it tests	Main confound
frozen k-NN	local embedding geometry	distance, normalization, and `k`
linear probe	linearly accessible task information	probe optimizer and regularization
shallow nonlinear probe	information available to a limited readout	probe capacity
partial adaptation	adaptability of selected layers	layer choice and schedule
full fine-tuning	end-to-end task performance	can erase or overcome pretraining differences
few-shot episodes	rapid adaptation under a task distribution	episode construction and transductive access
retrieval	cross-example or cross-modal neighbourhoods	candidate pool and duplicate leakage
transfer suite	breadth across tasks/domains	dataset selection and aggregate weighting

Freeze the encoder checkpoint before target labels are used. Tune probes equally and report both frozen and adapted performance when the claim concerns representation quality. Inspect nuisance predictability, calibration, subgroups, domain shift, and sample efficiency. Centred kernel alignment or representation similarity can compare geometries, but similarity is not functional equivalence.

Generative modelling: one target, several factorizations

Let data follow an unknown distribution p_data(x). A generative model may seek a normalized likelihood pθ(x), an implicit sampler, a latent-variable lower bound, a score field, or a time-dependent transport. Ask which mathematical quantity is available and which application actually matters.

Autoregressive models

The chain rule factorizes a joint distribution:

pθ(x) = ∏_i pθ(x_i | x_1,...,x_(i-1)).

Training by teacher-forced maximum likelihood is tractable and gives a normalized likelihood. Sampling is sequential in the chosen order, and exposure to self-generated prefixes differs from training. PixelRNN/PixelCNN apply this to images; decoder-only Transformers apply it to token sequences. Tokenization and order define the modelled space.

Variational autoencoders

For latent z, the marginal likelihood integrates pθ(x|z)p(z). A variational encoder qφ(z|x) gives the evidence lower bound (ELBO):

log pθ(x) ≥ E_q[log pθ(x|z)] - KL(qφ(z|x) || p(z)).

The reparameterization trick permits low-variance pathwise gradients for suitable continuous distributions. Reconstruction term, prior, posterior family, decoder capacity, and KL weighting shape the latent representation. Posterior collapse can occur when a powerful decoder ignores z. A higher ELBO does not guarantee more realistic or more useful samples.

Generative adversarial networks

A standard GAN solves a two-player objective:

min_G max_D  E_x[log D(x)] + E_z[log(1-D(G(z)))].

The discriminator supplies a learned training signal; the generator need not expose a normalized likelihood. In practice, non-saturating losses, Wasserstein objectives, gradient penalties, spectral normalization, and architecture choices affect stability. Mode dropping, oscillation, discriminator overfitting, and memorization require direct diagnostics.

Normalizing and continuous flows

An invertible map x = fθ(z) gives exact density by change of variables:

log p_X(x) = log p_Z(fθ⁻¹(x)) + log |det J_(fθ⁻¹)(x)|.

Coupling layers make inverse and log-determinant computation tractable. Invertibility constrains dimensions and architecture. Continuous normalizing flows define an ODE and accumulate a trace term. Exact likelihood is valuable but can rank perceptually different or out-of-distribution inputs counterintuitively.

Energy, score, and diffusion models

An energy model defines unnormalized density proportional to exp(-Eθ(x)); the partition function is generally difficult. Score matching learns ∇_x log p_t(x) without that normalizer.

Diffusion models gradually add noise in a known forward process and learn a reverse denoising process. DDPM training can be written as a weighted noise-, clean-data-, or velocity-prediction objective. Sampling quality and speed depend on noise schedule, parameterization, guidance, solver, and number of evaluations.

Elucidating the Design Space of Diffusion-Based Generative Models separates preconditioning, noise-level sampling, loss weighting, network design, and the numerical sampler. Treat these as independently controlled axes rather than one named model, and compare function evaluations as well as wall-clock cost.

Classifier-free guidance combines conditional and unconditional predictions to increase conditional alignment, often trading diversity for fidelity. Latent diffusion moves denoising into an autoencoder space, adding reconstruction and latent scaling assumptions.

Flow matching and few-step generation

Flow matching regresses a time-dependent vector field along chosen probability paths and samples by integrating an ODE. Rectified paths can reduce transport curvature. Consistency models train outputs at different noise levels to agree, enabling one- or few-step generation. MeanFlow learns an average velocity over an interval to target one-step or few-step transport.

One-step claims must match architecture, data, conditioning, distillation access, training compute, function evaluations, resolution, and metric protocol. Fast sampling does not by itself establish likelihood, coverage, or non-memorization.

The route now separates four mechanisms that are often collapsed under “flow”:

continuous transport: Flow Matching learns a velocity on a probability path, while rectified-flow work studies straighter paths and fewer numerical steps;
conditional paths and endpoint coupling: Generalized Conditional Flow Matching with minibatch optimal transport separates the conditional path family from how source and target examples are paired; OT-CFM uses minibatch optimal-transport couplings to reduce path complexity, but the minibatch coupling is not the population optimum;
sampler or trajectory distillation: Progressive Distillation repeatedly halves a diffusion sampler’s steps, so teacher access and accumulated training compute belong in every speed comparison; and
discrete probability flow: Discrete Flow Matching transports categorical mass with a continuous-time Markov chain rather than applying an image-space ODE to token IDs.

Scaling Rectified Flow Transformers connects the transport target to large text-to-image transformer design; LLaDA tests masked diffusion at language-model scale; and STFlow applies flow matching to spatial-transcriptomics prediction from histology. These are application branches of the maintained transport lineage, not reasons to create separate dated categories.

Evaluating generative models

Use a portfolio tied to the claim:

held-out log likelihood, bits per dimension, or an ELBO when mathematically available;
sample fidelity and coverage, not one alone;
precision/recall or density/coverage estimates in a declared feature space;
FID with the exact feature extractor, preprocessing, reference split, and sample count;
conditional accuracy using independent evaluators and human review where appropriate;
nearest-neighbour and membership tests for memorization/privacy;
subgroup and rare-mode coverage;
downstream utility evaluated without training/test contamination;
inference steps, latency, peak memory, training compute, and energy boundary.

Inception-based metrics inherit the feature extractor’s domain. They are especially unsafe as the only metric outside natural images. Human preference studies require randomization, blinding where possible, clear questions, rater counts, and uncertainty. The paper that introduced FID also studies a two-time-scale GAN update rule; do not attribute improvements to the metric or optimizer without separating those contributions.

Staged primary-source reading ladders

Representation learning

Order	Topic	Primary paper · year · venue/status	Read it for
1	transfer	How transferable are features in deep neural networks? — 2014 · NeurIPS	layer specialization and controlled source-to-target transfer
2	distillation	Distilling the Knowledge in a Neural Network — 2015 · arXiv workshop paper	soft targets, temperature, and teacher–student compression
3	domain adaptation	Domain-Adversarial Training of Neural Networks — 2016 · JMLR	gradient reversal and domain-invariant feature learning
4	metric meta-learning	Prototypical Networks for Few-shot Learning — 2017 · NeurIPS	episodic prototypes, learned metrics, and support/query separation
5	gradient meta-learning	Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks — 2017 · ICML	optimization-based few-step adaptation
6	continual retention	Overcoming Catastrophic Forgetting in Neural Networks — 2017 · PNAS	Fisher-weighted parameter consolidation and its stability–plasticity trade-off
7	queued contrast	Momentum Contrast for Unsupervised Visual Representation Learning — 2020 · CVPR	momentum keys and a negative queue decoupled from batch size
8	contrastive SSL	A Simple Framework for Contrastive Learning of Visual Representations — 2020 · ICML	SimCLR augmentations, projection heads, and batch negatives
9	supervised contrast	Supervised Contrastive Learning — 2020 · NeurIPS	multiple same-class positives and a controlled comparison with cross-entropy
10	semi-supervision	FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence — 2020 · NeurIPS	confidence-filtered pseudo-labels and strong augmentation
11	non-contrastive SSL	Bootstrap Your Own Latent — 2020 · NeurIPS	online/target asymmetry and collapse avoidance without negatives
12	redundancy reduction	Barlow Twins — 2021 · ICML	cross-correlation identity targets for invariance and redundancy reduction
13	self-distillation	Emerging Properties in Self-Supervised Vision Transformers — 2021 · ICCV	DINO centring, sharpening, and spatial representations
14	explicit anti-collapse	VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning — 2022 · ICLR	separate invariance, variance, and covariance terms
15	masked modelling	Masked Autoencoders Are Scalable Vision Learners — 2022 · CVPR	high-ratio masking and asymmetric reconstruction
16	parameter efficiency	LoRA: Low-Rank Adaptation of Large Language Models — 2022 · ICLR	frozen weights plus low-rank parameter updates
17	latent prediction	Self-Supervised Learning From Images With a Joint-Embedding Predictive Architecture — 2023 · CVPR	I-JEPA context-to-target latent prediction without pixel reconstruction
18	continual plasticity	Loss of plasticity in deep continual learning — 2024 · Nature	degradation during non-stationary learning and mechanisms that preserve adaptability

Generative modelling

Order	Family	Primary paper · year · venue/status	Read it for
1	unnormalized density	Estimation of Non-Normalized Statistical Models by Score Matching — 2005 · JMLR	learning a score without evaluating the partition function
2	variational latent model	Auto-Encoding Variational Bayes — 2014 · ICLR	the ELBO and reparameterized variational learning
3	adversarial model	Generative Adversarial Nets — 2014 · NeurIPS	implicit generation as a two-player game
4	autoregressive image model	Pixel Recurrent Neural Networks — 2016 · ICML	exact image likelihood through spatial factorization
5	invertible flow	Density estimation using Real NVP — 2017 · ICLR	affine coupling and tractable change of variables
6	adversarial distance	Wasserstein Generative Adversarial Networks — 2017 · ICML	integral probability metrics, critic constraints, and training signal
7	generative evaluation	GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium — 2017 · NeurIPS	TTUR and the original FID definition, assumptions, and protocol
8	continuous flow	Neural Ordinary Differential Equations — 2018 · NeurIPS	continuous-depth dynamics and continuous normalizing flows
9	expressive discrete flow	Glow: Generative Flow with Invertible 1×1 Convolutions — 2018 · NeurIPS	learned channel mixing in a tractable multiscale flow
10	noise-conditioned score	Generative Modeling by Estimating Gradients of the Data Distribution — 2019 · NeurIPS	denoising score matching across noise scales
11	diffusion	Denoising Diffusion Probabilistic Models — 2020 · NeurIPS	forward noising, learned reverse transitions, and variational training
12	score dynamics	Score-Based Generative Modeling through Stochastic Differential Equations — 2021 · ICLR	reverse-time SDEs, probability-flow ODEs, and score learning
13	faster diffusion path	Denoising Diffusion Implicit Models — 2021 · ICLR	non-Markovian forward processes and deterministic accelerated sampling
14	conditional guidance	Classifier-Free Diffusion Guidance — 2022 · arXiv; NeurIPS workshop origin	joint conditional/unconditional training and the fidelity–diversity trade
15	latent diffusion	High-Resolution Image Synthesis With Latent Diffusion Models — 2022 · CVPR	autoencoder-space diffusion and cross-attention conditioning
16	diffusion design	Elucidating the Design Space of Diffusion-Based Generative Models — 2022 · NeurIPS	preconditioning, loss weighting, noise schedules, and sampler choices as separable axes
17	straight transport	Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow — 2023 · ICLR	straightened probability paths and fewer integration steps
18	transport	Flow Matching for Generative Modeling — 2023 · ICLR	simulation-free vector-field regression on probability paths
19	conditional / OT coupling	Improving and Generalizing Flow-Based Generative Models with Minibatch Optimal Transport — 2024 · TMLR	conditional path families, independent versus minibatch-OT endpoint couplings, and trajectory complexity
20	few-step generation	Consistency Models — 2023 · ICML	self-consistency across noise scales and one/few-step sampling
21	sampler distillation	Progressive Distillation for Fast Sampling of Diffusion Models — 2022 · ICLR	repeated step halving, teacher-generated targets, and total training-cost accounting
22	scaled rectified flow	Scaling Rectified Flow Transformers for High-Resolution Image Synthesis — 2024 · ICML	rectified-flow targets, multimodal diffusion-transformer blocks, and high-resolution text-to-image scaling
23	discrete transport	Discrete Flow Matching — 2024 · NeurIPS	categorical probability paths, posterior parameterizations, schedules, and non-autoregressive sampling
24	discrete language diffusion	Large Language Diffusion Models — 2025 · NeurIPS	masked diffusion pretraining and reversal-law analysis at language-model scale
25	mean transport	Mean Flows for One-step Generative Modeling — 2025 · NeurIPS	interval-average velocity fields and current one-step generation evidence
26	histology-to-spatial flow	STFlow — 2025 · ICML	continuous transport from histology-conditioned representations to spatial gene-expression distributions

Controlled labs

Lab A: representation protocol

Use one fixed image dataset and split. Pretrain a small encoder using supervised classification, SimCLR, and a masked autoencoder under matched images, updates, architecture scale, augmentations where applicable, and tuning trials.

Freeze each checkpoint before accessing target labels.
Evaluate k-NN, a tuned linear probe, and full fine-tuning.
Repeat with 1%, 10%, and 100% of target labels.
Test a shifted colour or acquisition domain and a nuisance-prediction probe.
Report paired intervals, calibration, compute, and representation collapse metrics.

Lab B: generative mechanisms

Start with a two-dimensional mixture whose true density and modes are visible. Compare a VAE, GAN, normalizing flow, diffusion model, and flow-matching model. Then repeat a small version on Fashion-MNIST or CIFAR-10.

Hold data, preprocessing, backbone capacity range, training examples, selection budget, seeds, and metric implementation fixed. Report held-out likelihood/ELBO when available, mode coverage, fidelity, nearest training neighbours, function evaluations, latency, and peak memory. Do not compare only the most attractive sample grid.

Common failure modes

Calling an augmentation invariant without checking whether it changes the label.
Leaking downstream labels into encoder, checkpoint, or hyperparameter selection.
Giving one representation a stronger probe or larger fine-tuning budget.
Treating a linear probe as the only definition of representation quality.
Assuming domain alignment preserves class information or causal mechanisms.
Comparing few-shot methods with unequal pretraining data or transductive access.
Reporting trainable parameter count while hiding activation or optimizer memory.
Interpreting VAE reconstruction quality as exact likelihood or semantic disentanglement.
Evaluating GANs or diffusion models with FID alone.
Claiming no mode collapse from a finite sample grid.
Using a natural-image feature metric on another domain without validation.
Calling a one-step generator superior without matching training compute and quality.
Omitting duplicate and memorization checks for publicly released generators.

Self-check

Without notes, answer:

What invariance does a positive-pair construction impose?
How do SimCLR, BYOL, DINO, and MAE obtain their targets?
Why can a frozen linear probe and full fine-tuning rank encoders differently?
What assumptions make domain-adversarial alignment useful or harmful?
Derive the VAE ELBO from a KL divergence.
Why can a GAN sample without providing a normalized likelihood?
Which architectural constraint makes a normalizing flow’s density tractable?
What function does a score model learn?
How do diffusion and flow matching define paths between noise and data?
Why must fidelity, coverage, memorization, and inference cost be reported separately?

Exit task

Choose one representation method and one generative method relevant to your field. Reconstruct each objective, target construction, architecture, and evaluation assumption. Run a small equal-budget comparison with frozen splits and seeds. Report downstream sample efficiency for the representation and likelihood/proxy, coverage, fidelity, memorization, and cost for the generator. State one conclusion that the results support and two they do not.

Resolution transfer and calibrated geophysical fields

BaguanHR creates additional fine-resolution atmospheric training fields through super-resolution, whereas Sparse2Cloud predicts dense cloud geometry from sparse active-sensor tracks and calibrates quantile intervals. Compare transferred information, geographic or temporal splits and uncertainty, not just grid density.

Primary paper · year · evidence	Core idea and why to read it	Official resources
Pushing the Limits of High-Resolution Weather Forecasting through Data ScalingYang Zhao, Peisong Niu, Tian Zhou, Ziqing Ma, Guanlong Ma, Rong Jin, Huiling Yuan, and Liang Sun2026 · ECCV · acceptedECCV 2026 Spotlight verified in the official program, session Computational, Medical & Scientific Imaging; scientific claims use conference-hosted manuscript #7258. Publisher proceedings/version of record was not verified on 2026-09-12.	BaguanHR transfers high-resolution data rather than merely transferring a coarse forecast model. Variable-specific super-resolution expands scarce 0.1-degree atmospheric training data from long 0.25-degree ERA5 records, then a high-resolution forecaster learns from synthetic-plus-real fields. Adaptive rollout weighting and replay target long-horizon error accumulation.	ECCV Spotlight program · Conference manuscript · Author project (paper/program link)

Reading question: Does a finer output represent measured extra information, learned prior interpolation or a calibrated uncertainty statement?

Evidence boundary: Synthetic resolution does not guarantee missing fine physics. Marginal conformal coverage assumes exchangeability and is not a universal conditional guarantee under climate or geographic shift.

Implicit-field tokenization of biomedical graphs

VesselTok compresses pre-extracted centerline graphs through pseudo-radius occupancy reconstruction, then uses latent generative models for synthesis and repair. This connects shape representation, VAE bottlenecks and graph topology without claiming to infer vessels directly from raw CT.

Primary paper · year · evidence	Core idea and why to read it	Official resources
VesselTok: Tokenizing Vessel-like 3D Biomedical Graph Representations for Reconstruction and GenerationChinmay Prabhakar, Bastian Wittmann, Tamaz Amiranashvili, Paul Büschl, Ezequiel De la Rosa, Julian McGinnis, Benedikt Wiestler, Bjoern Menze, and Suprosanna Shit2026 · ECCV · acceptedECCV 2026 Spotlight verified in the official program, session Computational, Medical & Scientific Imaging; scientific claims use conference-hosted manuscript #10927. Publisher proceedings/version of record was not verified on 2026-09-12.	VesselTok compresses complex biomedical centerline graphs into continuous latent tokens by reconstructing a pseudo-radius occupancy field. Centerline-conditioned attention preserves thin branching structure more efficiently than densely sampling its surface. The same latent supports graph generation and conditional repair, but it does not directly encode true vessel thickness.	ECCV Spotlight program · Conference manuscript · Official implementation

Reading question: What geometric and topological information survives token compression, and which attributes were never represented?

Evidence boundary: Pseudo radius is not measured vessel thickness; high overlap can coexist with worse cycle topology, and token capacity can merge or omit small structures.

Low-resolution student inference versus resolution-cascaded generation

GAP-Track removes teacher and residual reconstruction branches at inference. RMD instead retains high-resolution generative refinement. Both exploit cross-resolution supervision but solve different deployment problems.

Primary paper · year · evidence	Core idea and why to read it	Official resources
Cross-Resolution Distribution Matching for Diffusion DistillationFeiyang Chen, Hongpeng Pan, Haonan Xu, Xinyu Duan, Zhefeng Wang, and Yang Yang2026 · ECCV · acceptedECCV 2026 poster verified from the official linked program event. Review uses the conference-hosted manuscript checked 2026-09-12; a separate Springer version of record was not verified. RMD retains high-resolution refinement at inference; it is not low-resolution-only recognition. Reported speedups depend on the single-A100 baseline, CFG and NFE. The conference PDF author order differs from the arXiv record and is preserved here.	RMD distills a diffusion generator into a few-step trajectory that begins at low resolution and ends at high resolution. It aligns resolution-specific noise states using logSNR, projects student distributions into the teacher’s resolution, and uses distribution matching plus noise reinjection to train the cascade. The output remains a high-resolution generated image or video.	Official ECCV poster record

Reading question: Is high resolution absent at deployment, or only used for fewer steps? What is the full-resolution reference?

Evidence boundary: Synthetic downsampling does not reproduce all optical changes between microscope magnifications; lower-resolution gains need not equal full-resolution performance.

Masked generation and diffusion-friendly latent spectra

MIMFlow combines masked image modeling with normalizing flows; SSVAE changes the spectral structure of video latents to improve downstream diffusion. Separate tokenizer reconstruction quality, likelihood or generation objectives, and final sample quality.

Primary paper · year · evidence	Core idea and why to read it	Official resources
MIMFlow: Integrating Masked Image Modeling with Normalizing Flows for End-to-End Image GenerationYang Chen, Xiaowei Xu, Shuai Wang, Xinwen Zhang, Qiushi Guo, Tiezheng Ge, and Limin Wang2026 · ECCV · acceptedECCV 2026 Oral verified in the official program; scientific claims use the conference-hosted manuscript #2272. Publisher proceedings/version of record was not verified on 2026-09-12.	MIMFlow makes a masked visual tokenizer and its normalizing-flow prior learn together. A fixed 128-token bottleneck extracts global information from incomplete images; the invertible flow models those semantic latents, while a separate decoder reconstructs detailed pixels. The important change is where modeling capacity is spent, not simply making a larger flow.	ECCV Oral program · Conference manuscript · Official implementation · arXiv manuscript
Delving into Latent Spectral Biasing of Video VAEs for Superior DiffusabilityShizhan Liu, Xinran Deng, Zhuoyi Yang, Jiayan Teng, Xiaotao Gu, and Jie Tang2026 · ECCV · acceptedECCV 2026 Oral verified in the official program; scientific claims use the conference-hosted manuscript #7577. Publisher proceedings/version of record was not verified on 2026-09-12. The initial arXiv manuscript appeared in December 2025; year follows the 2026 conference.	SSVAE trains video latents to be easier for a diffusion model to learn, not merely easy for a decoder to reconstruct. It promotes two separate spectral biases: a few dominant channel modes and greater low-frequency spatio-temporal energy. Latent Masked Reconstruction (LMR) and Local Correlation Regularization (LCR) implement those targets without requiring a new downstream diffusion objective.	ECCV Oral program · Conference manuscript · Official implementation · arXiv manuscript · Model checkpoints

Reading question: Which representation is easier to model, and what fidelity or sampling cost is exchanged for that advantage?

Evidence boundary: Good reconstruction alone does not establish good diffusability, and normalizing flows are not interchangeable with flow matching.

Causal audiovisual generation and variable-step distillation

OmniForcing transfers a bidirectional audio-video generator to causal streaming; AnyFlow learns on-policy flow maps supporting different sampling budgets. Preserve training-stage, self-rollout, synchronization, step-count and hardware qualifiers.

Primary paper · year · evidence	Core idea and why to read it	Official resources
AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map DistillationYuchao Gu, Guian Fang, Yuxin Jiang, Weijia Mao, Song Han, Han Cai, and Mike Zheng Shou2026 · ECCV · acceptedECCV 2026 Spotlight verified in the official program; scientific claims use the conference-hosted manuscript #9062. Publisher proceedings/version of record was not verified on 2026-09-12.	AnyFlow distills a video generator that can use different sampling budgets without retraining. Instead of learning only a noisy-state-to-clean endpoint map, it learns transitions between two arbitrary diffusion times. A second, on-policy stage trains on the student's own trajectories using shortcut flow-map simulation, improving few-step quality while retaining useful refinement at more steps.	ECCV Spotlight program · Conference manuscript · Official implementation · arXiv manuscript · Author project · Model collection

Reading question: What teacher information disappears at inference, and does quality remain stable as the generated history or step budget changes?

Evidence boundary: Real-time and any-step claims depend on resolution, duration, hardware and guidance; source-code revisions may change the training recipe.

Volume tokens, persistent geometry and registration

Compare learned volume representations, streaming geometric context, flow-based rigid registration and dense image matching. Each chooses a different correspondence unit and computation path; learned priors complement rather than abolish geometric constraints.

Primary paper · year · evidence	Core idea and why to read it	Official resources
Register Any Point: Scaling 3D Point Cloud Registration by Flow MatchingYue Pan, Tao Sun, Liyuan Zhu, Lucas Nunes, Iro Armeni, Jens Behley, and Cyrill Stachniss2026 · ECCV · acceptedAccepted to ECCV 2026; oral presentation verified in the official program. This record and review use the conference-hosted manuscript checked 2026-09-12; a separate Springer proceedings record was not verified. The arXiv preprint first appeared in 2025; the catalog year follows its verified 2026 conference acceptance.	RAP generates the registered configuration of several point clouds directly with conditional flow matching, then recovers rigid poses from that configuration. Its crucial addition is test-time rigidity enforcement: each integration step projects the predicted clean points back to a rigid transform of each input view, reducing the mismatch between unconstrained point motion and a rigid registration task.	Official ECCV oral presentation · Official ECCV paper record · arXiv record · Official code · Official project

Reading question: Which coordinates, features or correspondences are predicted, and how are rigidity, memory growth and distribution shift tested?

Evidence boundary: Compact per-frame context still accumulates; multi-step integration is not one-pass inference, and improved average matching can hide benchmark regressions.

Real-data correction in generative distillation

RealUID unifies matching-model distillation and routes real examples through an auxiliary matching model. Read the alternating optimization and coefficient assumptions before interpreting the one-step generator results.

Primary paper · year · evidence	Core idea and why to read it	Official resources
Universal Inverse Distillation for Matching Models with Real-Data Supervision (No GANs)Nikita Kornilov, David Li, Tikhon Mavrin, Aleksei Leonov, Nikita Gushchin, Evgeny Burnaev, Iaroslav Koshelev, Aleksandr Korotin2026 · ICLR · peer-reviewedOfficial ICLR 2026 oral and full proceedings manuscript checked. Three-network training yields a one-step generator. No GAN discriminator does not mean no min–max optimization. Selected low-resolution image results include additional fine-tuning; teacher-error correction has restrictive assumptions. No cross-magnification or pathology evaluation.	RealUID adds real-data supervision to a unified inverse-distillation formulation for matching models. A frozen teacher, a one-step generator and a learned fake matching model interact through matching losses. No GAN discriminator is added, but alternating min–max optimization remains: “No GANs” does not mean no adversarially structured optimization.	Official proceedings record · OpenReview

Reading question: Where does real data enter the teacher–student system, and which extra training stages are included in the comparison?

Evidence boundary: No GAN discriminator does not mean no min–max problem; low-resolution generation does not establish pathology validity.

Distributional geometry of contrastive representations

InfoNCE Induces Gaussian Distribution separates a plateau/projection argument from an explicitly regularized surrogate. Its diagnostic table is a lesson in testing, rather than assuming, representation distributions.

Primary paper · year · evidence	Core idea and why to read it	Official resources
InfoNCE Induces Gaussian DistributionRoy Betser, Eyal Gofer, Meir Yossef Levi, Guy Gilboa2026 · ICLR · peer-reviewedOfficial ICLR 2026 oral and full proceedings manuscript checked. Conditional theory and representation diagnostics, not proof that every InfoNCE encoder is jointly Gaussian. Separate fixed-dimensional projection limits from a regularized population surrogate and coordinate-wise non-rejection from joint normality.	This paper investigates when contrastive representations have approximately Gaussian low-dimensional marginals. It gives two different analytical routes: an alignment/uniformity argument under plateau and concentration assumptions, and an explicitly regularized population surrogate. The title is broader than a theorem that arbitrary InfoNCE training always produces jointly Gaussian embeddings.	Official proceedings record · OpenReview

Reading question: Which assumptions make the distributional claim possible, and what does a univariate normality test actually establish?

Evidence boundary: Coordinate-wise non-rejection is not joint Gaussianity; the paper does not prove that arbitrary SGD reaches the described optimum.

Predictive representations for reward-selected control

TD-JEPA connects non-contrastive latent prediction to temporal-difference learning. Read the state/action input contract and downstream task-vector inference before interpreting zero-shot control.

Primary paper · year · evidence	Core idea and why to read it	Official resources
TD-JEPA: Latent-predictive Representations for Zero-Shot Reinforcement LearningMarco Bagatella, Matteo Pirotta, Ahmed Touati, Alessandro Lazaric, Andrea Tirinzoni2026 · ICLR · peer-reviewedOfficial ICLR 2026 oral and full proceedings manuscript checked. Reward-free state–action transitions, not action-free video. Zero-shot policy selection still requires reward observations for task-vector inference. Theoretical assumptions, benchmark-specific actor inputs and mixed OGBench comparisons are qualified in the source-checked review; no WSI evaluation.	TD-JEPA learns state and task representations from reward-free transitions so that a new reward can select a policy without task-specific policy fine-tuning. Its distinguishing move is temporal-difference bootstrapping in a joint-embedding predictive architecture: predicting accumulated future task features connects representation learning to zero-shot control.	Official proceedings record · OpenReview · Official implementation

Reading question: What future quantity is predicted, what prevents collapse, and what reward information is still needed at deployment?

Evidence boundary: Restricted theory and simulator-specific actor inputs do not prove arbitrary video or pathology-control validity.

Generative geometry: separate pairing cost, endpoint normalization and trajectory

SOT-CFM changes optimal-transport pairing to an angular cost. Spherical Flow Matching additionally places noise and data on a common sphere and regresses geodesic tangent velocities. These interventions expose whether the gain comes from pairing, removed radial variation, or the path itself.

Primary paper · year · evidence	Core idea and why to read it	Official resources
Geometry-Aware Image Flow MatchingJunho Lee, Kwanseok Kim, and Joonseok Lee2026 · ICML · peer-reviewed	This paper asks whether image generation can benefit from separating direction from magnitude. SOT-CFM changes the optimal-transport pairing cost to an angular distance; Spherical Flow Matching additionally projects both endpoints to a sphere and learns geodesic tangent dynamics. These are two different interventions, not two names for the same normalization trick.	Final proceedings PDF · Author manuscript

Reading question: Which geometric assumption changes, and what information could normalization discard?

Evidence boundary: ImageNet results use separately tuned guidance and radius; precision need not improve with gFID. Extreme norms and domains where magnitude carries meaning require explicit checks before transfer.

Global semantic anchors and local patch targets

Two Global Crops Suffice separates same-image global-view alignment from local crops and iBOT refinement in controlled DINO-family experiments. CODI instead changes the patch target: weakly mix the same teacher view’s class distribution into its local patch distribution. The diagnostic study and target-side method answer related questions, but neither independently validates the other.

Primary paper · year · evidence	Core idea and why to read it	Official resources
Two Global Crops Suffice: Locating Semantic Emergence in DINO-Style Self-Supervised LearningBasavaraj Sunagad, Artur Jesslen, and Adam Kortylewski2026 · arXiv · preprintarXiv:2609.28187v1, submitted 23 September 2026, inspected 26 October 2026. Empirical DINO-family ablations on ImageNet-1k, not a universal necessity theorem or an evaluation of DINOv3/pathology. The review retains a discrepancy between the main text’s fixed-scale depth claim and Appendix Table A3.	This is a controlled diagnosis of DINO-style learning, not a new foundation model. In the tested ImageNet-1k regime, matching two geometrically different global crops from the same image already produces semantic patch correspondence. iBOT’s masked-patch objective improves that structure when combined with the global objective, while extra local crops chiefly improve classification in the DINO-only comparison. The practical lesson is to separate semantic anchoring, refinement and evaluation rather than treat a successful recipe as an indivisible block.	Inspected manuscript and appendices (v1)
CODI: Improving Self-Supervised Vision Transformers with Cross DistillationSiran Dai, Qianqian Xu, Peisong Wen, Yang Liu, Xiaochun Cao, and Qingming Huang2026 · arXiv · preprintarXiv:2610.30639v1, submitted 22 October 2026, inspected 26 October 2026. The manuscript has a NeurIPS 2026 footer, but no official acceptance record was independently located in this audit; retained as a preprint. The author repository contains only a Coming soon README, not released implementation code.	CODI changes a masked patch’s teaching target, not the Vision Transformer architecture. It mixes 90% of the momentum teacher’s local patch distribution with 10% of that same view’s global class-token distribution. This gives low-attention patches a direct semantic training signal while retaining mostly local supervision. It is self-distillation during unlabeled image pretraining, not compression from a separately pretrained large teacher and not cross-magnification transfer.	Inspected manuscript (v1) · Author repository — implementation pending at inspection

Reading question: Which signal creates useful spatial semantics, which refines it, and do your global and dense evaluations agree?

Evidence boundary: Both inspected sources are preprints. Crop ablations match iterations, not all computation; the depth-scale narrative has an appendix exception. CODI’s gradient statement is local/logit-level, its alpha is task-dependent, and its code was not released at inspection. Neither reports WSI or cross-magnification validation.

Complete maintained paper route

65 canonical readings · 2005–2026· route reviewed through October 2, 2026.

This is the complete structured list for this chapter, ordered by an explicit prerequisite-aware teaching sequence. The explanatory ladder above highlights the shortest conceptual route; this list also includes important branches, benchmarks, and current endpoints. It is generated from the same deduplicated catalog used by the AI background registry and My Library, so a future catalog update appears here automatically.

Show all 65 readings in maintained order

#12014NeurIPSPeer reviewedL2
How Transferable Are Features in Deep Neural Networks?
Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson
Metadata checked 2026-07-27
#22015NIPS Deep Learning and Representation Learning WorkshopTechnical reportL1–L2
Distilling the Knowledge in a Neural Network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean
Metadata checked 2026-07-27
#32016JMLRPeer reviewedL2
Domain-Adversarial Training of Neural Networks
Yaroslav Ganin et al.
Metadata checked 2026-07-27
#42017NeurIPSPeer reviewedL2
Prototypical Networks for Few-shot Learning
Jake Snell, Kevin Swersky, and Richard Zemel
Metadata checked 2026-07-28
#52017ICMLPeer reviewedL2–L3
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Chelsea Finn, Pieter Abbeel, and Sergey Levine
Metadata checked 2026-07-27
#62017Proceedings of the National Academy of SciencesPeer reviewedL2
Overcoming Catastrophic Forgetting in Neural Networks
James Kirkpatrick et al.
Metadata checked 2026-07-28
#72020CVPRPeer reviewedResearch
Momentum Contrast for Unsupervised Visual Representation Learning
Kaiming He et al.
Metadata checked 2026-07-27
#82020ICMLPeer reviewedL2
A Simple Framework for Contrastive Learning of Visual Representations
Ting Chen et al.
Metadata checked 2026-07-27
#92026ICLRPeer reviewedL3
InfoNCE Induces Gaussian Distribution
Roy Betser, Eyal Gofer, Meir Yossef Levi, Guy Gilboa
Official ICLR 2026 oral and full proceedings manuscript checked. Conditional theory and representation diagnostics, not proof that every InfoNCE encoder is jointly Gaussian. Separate fixed-dimensional projection limits from a regularized population surrogate and coordinate-wise non-rejection from joint normality.
Official artifacts: Official proceedings record · OpenReview
Metadata checked 2026-09-19
#102020NeurIPSPeer reviewedL2
Supervised Contrastive Learning
Prannay Khosla et al.
Official artifacts: Code
Metadata checked 2026-07-28
#112020NeurIPSPeer reviewedL2
FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
Kihyuk Sohn et al.
Metadata checked 2026-07-27
#122020NeurIPSPeer reviewedResearch
Bootstrap Your Own Latent
Jean-Bastien Grill et al.
Metadata checked 2026-07-27
#132026ICLRPeer reviewedL3
TD-JEPA: Latent-predictive Representations for Zero-Shot Reinforcement Learning
Marco Bagatella, Matteo Pirotta, Ahmed Touati, Alessandro Lazaric, Andrea Tirinzoni
Official ICLR 2026 oral and full proceedings manuscript checked. Reward-free state–action transitions, not action-free video. Zero-shot policy selection still requires reward observations for task-vector inference. Theoretical assumptions, benchmark-specific actor inputs and mixed OGBench comparisons are qualified in the source-checked review; no WSI evaluation.
Official artifacts: Official proceedings record · OpenReview · Official implementation
Metadata checked 2026-09-19
#142021ICMLPeer reviewedResearch
Barlow Twins: Self-Supervised Learning via Redundancy Reduction
Jure Zbontar et al.
Metadata checked 2026-07-27
#152021ICCVPeer reviewedL2
Emerging Properties in Self-Supervised Vision Transformers
Mathilde Caron et al.
Metadata checked 2026-07-27
#162026arXivPreprintL3
Two Global Crops Suffice: Locating Semantic Emergence in DINO-Style Self-Supervised Learning
Basavaraj Sunagad, Artur Jesslen, and Adam Kortylewski
arXiv:2609.28187v1, submitted 23 September 2026, inspected 26 October 2026. Empirical DINO-family ablations on ImageNet-1k, not a universal necessity theorem or an evaluation of DINOv3/pathology. The review retains a discrepancy between the main text’s fixed-scale depth claim and Appendix Table A3.
Official artifacts: Inspected manuscript and appendices (v1)
Metadata checked 2026-10-26
#172026arXivPreprintL3
CODI: Improving Self-Supervised Vision Transformers with Cross Distillation
Siran Dai, Qianqian Xu, Peisong Wen, Yang Liu, Xiaochun Cao, and Qingming Huang
arXiv:2610.30639v1, submitted 22 October 2026, inspected 26 October 2026. The manuscript has a NeurIPS 2026 footer, but no official acceptance record was independently located in this audit; retained as a preprint. The author repository contains only a Coming soon README, not released implementation code.
Official artifacts: Inspected manuscript (v1) · Author repository — implementation pending at inspection
Metadata checked 2026-10-26
#182022ICLRPeer reviewedResearch
VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning
Adrien Bardes, Jean Ponce, and Yann LeCun
Metadata checked 2026-07-27
#192022CVPRPeer reviewedL2
Masked Autoencoders Are Scalable Vision Learners
Kaiming He et al.
Metadata checked 2026-07-27
#202022ICLRPeer reviewedL2
LoRA: Low-Rank Adaptation of Large Language Models
Edward J. Hu et al.
Metadata checked 2026-07-27
#212023CVPRPeer reviewedResearch
Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture
Mahmoud Assran et al.
Metadata checked 2026-07-27
#222025arXivPreprintL3
LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics
Randall Balestriero and Yann LeCun
Preprint revised through 2026-05-25; no peer-reviewed version of record was located by 2026-08-30.
Official artifacts: Official implementation
Metadata checked 2026-08-30
#232026arXivPreprintL3
LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics
Lukas Kuhn, Lucas Maes, Giuseppe Serra, Quentin Le Lidec, Yann LeCun, Randall Balestriero, and Florian Buettner
arXiv v1 preprint submitted 27 August 2026; compute and transfer claims have not yet received proceedings review.
Official artifacts: Official project · Official implementation · Official model
Metadata checked 2026-08-30
#242024NaturePeer reviewedL2–L3
Loss of Plasticity in Deep Continual Learning
Shibhansh Dohare et al.
Metadata checked 2026-07-27
#252005JMLRPeer reviewedL2–L3
Estimation of Non-Normalized Statistical Models by Score Matching
Aapo Hyvärinen
Metadata checked 2026-07-27
#262014ICLRPeer reviewedL2
Auto-Encoding Variational Bayes
Diederik P. Kingma and Max Welling
Metadata checked 2026-07-27
#272026ECCVAccepted · proceedings pendingL3
VesselTok: Tokenizing Vessel-like 3D Biomedical Graph Representations for Reconstruction and Generation
Chinmay Prabhakar, Bastian Wittmann, Tamaz Amiranashvili, Paul Büschl, Ezequiel De la Rosa, Julian McGinnis, Benedikt Wiestler, Bjoern Menze, and Suprosanna Shit
ECCV 2026 Spotlight verified in the official program, session Computational, Medical & Scientific Imaging; scientific claims use conference-hosted manuscript #10927. Publisher proceedings/version of record was not verified on 2026-09-12.
Official artifacts: ECCV Spotlight program · Conference manuscript · Official implementation
Metadata checked 2026-09-12
#282014NeurIPSPeer reviewedL2
Generative Adversarial Nets
Ian Goodfellow et al.
Metadata checked 2026-07-27
#292016ICMLPeer reviewedL2
Pixel Recurrent Neural Networks
Aäron van den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu
Metadata checked 2026-07-27
#302017ICLRPeer reviewedL2–L3
Density Estimation using Real NVP
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio
Metadata checked 2026-07-27
#312017ICMLPeer reviewedL2–L3
Wasserstein Generative Adversarial Networks
Martin Arjovsky, Soumith Chintala, and Léon Bottou
Metadata checked 2026-07-27
#322017NeurIPSPeer reviewedL2
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter
Metadata checked 2026-07-28
#332018NeurIPSPeer reviewedL3
Neural Ordinary Differential Equations
Ricky T. Q. Chen et al.
Metadata checked 2026-07-27
#342018NeurIPSPeer reviewedL2–L3
Glow: Generative Flow with Invertible 1×1 Convolutions
Diederik P. Kingma and Prafulla Dhariwal
Metadata checked 2026-07-27
#352026ECCVAccepted · proceedings pendingL3
MIMFlow: Integrating Masked Image Modeling with Normalizing Flows for End-to-End Image Generation
Yang Chen, Xiaowei Xu, Shuai Wang, Xinwen Zhang, Qiushi Guo, Tiezheng Ge, and Limin Wang
ECCV 2026 Oral verified in the official program; scientific claims use the conference-hosted manuscript #2272. Publisher proceedings/version of record was not verified on 2026-09-12.
Official artifacts: ECCV Oral program · Conference manuscript · Official implementation · arXiv manuscript
Metadata checked 2026-09-12
#362019NeurIPSPeer reviewedL3
Generative Modeling by Estimating Gradients of the Data Distribution
Yang Song and Stefano Ermon
Metadata checked 2026-07-27
#372020NeurIPSPeer reviewedL2
Denoising Diffusion Probabilistic Models
Jonathan Ho, Ajay Jain, and Pieter Abbeel
Metadata checked 2026-07-27
#382021ICLRPeer reviewedL3
Score-Based Generative Modeling through Stochastic Differential Equations
Yang Song et al.
Metadata checked 2026-07-27
#392021ICLRPeer reviewedL2–L3
Denoising Diffusion Implicit Models
Jiaming Song, Chenlin Meng, and Stefano Ermon
Metadata checked 2026-07-27
#402022arXivPreprintL2–L3
Classifier-Free Diffusion Guidance
Jonathan Ho and Tim Salimans
Metadata checked 2026-07-27
#412022CVPRPeer reviewedL2–L3
High-Resolution Image Synthesis with Latent Diffusion Models
Robin Rombach et al.
Metadata checked 2026-07-27
#422026ECCVAccepted · proceedings pendingL3
Delving into Latent Spectral Biasing of Video VAEs for Superior Diffusability
Shizhan Liu, Xinran Deng, Zhuoyi Yang, Jiayan Teng, Xiaotao Gu, and Jie Tang
ECCV 2026 Oral verified in the official program; scientific claims use the conference-hosted manuscript #7577. Publisher proceedings/version of record was not verified on 2026-09-12. The initial arXiv manuscript appeared in December 2025; year follows the 2026 conference.
Official artifacts: ECCV Oral program · Conference manuscript · Official implementation · arXiv manuscript · Model checkpoints
Metadata checked 2026-09-12
#432022NeurIPSPeer reviewedL2–L3
Elucidating the Design Space of Diffusion-Based Generative Models
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine
Official artifacts: Code
Metadata checked 2026-07-28
#442022ICLRPeer reviewedL2–L3
Progressive Distillation for Fast Sampling of Diffusion Models
Tim Salimans and Jonathan Ho
Official artifacts: Official code
Metadata checked 2026-08-10
#452026ICLRPeer reviewedL3
Universal Inverse Distillation for Matching Models with Real-Data Supervision (No GANs)
Nikita Kornilov, David Li, Tikhon Mavrin, Aleksei Leonov, Nikita Gushchin, Evgeny Burnaev, Iaroslav Koshelev, Aleksandr Korotin
Official ICLR 2026 oral and full proceedings manuscript checked. Three-network training yields a one-step generator. No GAN discriminator does not mean no min–max optimization. Selected low-resolution image results include additional fine-tuning; teacher-error correction has restrictive assumptions. No cross-magnification or pathology evaluation.
Official artifacts: Official proceedings record · OpenReview
Metadata checked 2026-09-19
#462026ECCVAccepted · proceedings pendingL3
Cross-Resolution Distribution Matching for Diffusion Distillation
Feiyang Chen, Hongpeng Pan, Haonan Xu, Xinyu Duan, Zhefeng Wang, and Yang Yang
ECCV 2026 poster verified from the official linked program event. Review uses the conference-hosted manuscript checked 2026-09-12; a separate Springer version of record was not verified. RMD retains high-resolution refinement at inference; it is not low-resolution-only recognition. Reported speedups depend on the single-A100 baseline, CFG and NFE. The conference PDF author order differs from the arXiv record and is preserved here.
Official artifacts: Official ECCV poster record
Metadata checked 2026-09-12
#472026CVPRPeer reviewedL2–L3
LIFT and PLACE: A Simple, Stable, and Effective Knowledge Distillation Framework for Lightweight Diffusion Models
Hyunsoo Han, Sangyeop Yeo, and Jaejun Yoo
Metadata checked 2026-08-10
#482023ICLRPeer reviewedL3
Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
Xingchao Liu, Chengyue Gong, and Qiang Liu
Metadata checked 2026-07-27
#492023ICLRPeer reviewedL3
Flow Matching for Generative Modeling
Yaron Lipman et al.
Metadata checked 2026-07-27
#502026ECCVAccepted · proceedings pendingL3
Register Any Point: Scaling 3D Point Cloud Registration by Flow Matching
Yue Pan, Tao Sun, Liyuan Zhu, Lucas Nunes, Iro Armeni, Jens Behley, and Cyrill Stachniss
Accepted to ECCV 2026; oral presentation verified in the official program. This record and review use the conference-hosted manuscript checked 2026-09-12; a separate Springer proceedings record was not verified. The arXiv preprint first appeared in 2025; the catalog year follows its verified 2026 conference acceptance.
Official artifacts: Official ECCV oral presentation · Official ECCV paper record · arXiv record · Official code · Official project
Metadata checked 2026-09-12
#512024Transactions on Machine Learning ResearchPeer reviewedL2–L3
Improving and Generalizing Flow-Based Generative Models with Minibatch Optimal Transport
Alexander Tong, Kilian Fatras, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector-Brooks, Guy Wolf, and Yoshua Bengio
Official artifacts: Official code
Metadata checked 2026-08-10
#522026ICMLPeer reviewedL3
Geometry-Aware Image Flow Matching
Junho Lee, Kwanseok Kim, and Joonseok Lee
Official artifacts: Final proceedings PDF · Author manuscript
Metadata checked 2026-10-12
#532023ICMLPeer reviewedL3
Consistency Models
Yang Song et al.
Metadata checked 2026-07-27
#542024ICMLPeer reviewedL2–L3
Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
Patrick Esser et al.
Metadata checked 2026-08-10
#552024NeurIPSPeer reviewedL2–L3
Discrete Flow Matching
Itai Gat, Tal Remez, Neta Shaul, Felix Kreuk, Ricky T. Q. Chen, Gabriel Synnaeve, Yossi Adi, and Yaron Lipman
Metadata checked 2026-08-10
#562025NeurIPSPeer reviewedL3
Mean Flows for One-step Generative Modeling
Zhengyang Geng et al.
Metadata checked 2026-07-27
#572026ECCVAccepted · proceedings pendingL3
AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillation
Yuchao Gu, Guian Fang, Yuxin Jiang, Weijia Mao, Song Han, Han Cai, and Mike Zheng Shou
ECCV 2026 Spotlight verified in the official program; scientific claims use the conference-hosted manuscript #9062. Publisher proceedings/version of record was not verified on 2026-09-12.
Official artifacts: ECCV Spotlight program · Conference manuscript · Official implementation · arXiv manuscript · Author project · Model collection
Metadata checked 2026-09-12
#582025NeurIPSPeer reviewedL2–L3
Large Language Diffusion Models
Shen Nie et al.
Official artifacts: Official code and checkpoints
Metadata checked 2026-08-10
#592025ICMLPeer reviewedL3
Scalable Generation of Spatial Transcriptomics from Histology Images via Whole-Slide Flow Matching
Tinglin Huang, Tianyu Liu, Mehrtash Babadi, Wengong Jin, and Rex Ying
Official artifacts: Official code
Metadata checked 2026-08-10
#602026arXivPreprintL3
ReFlowSET: Representation-Aligned Latent Flow Matching for SAR-to-EO Image Translation
Jeonghyeok Do, Seungchul Lee, and Munchurl Kim
arXiv v1 submitted 1 September 2026. The exact title follows the current arXiv record, superseding an earlier feed title. Code, weights, and an official project page were available; no peer-reviewed record was verified.
Official artifacts: Official implementation and checkpoints · Official project page
Metadata checked 2026-09-02
#612026ECCVAccepted · proceedings pendingL3
Pushing the Limits of High-Resolution Weather Forecasting through Data Scaling
Yang Zhao, Peisong Niu, Tian Zhou, Ziqing Ma, Guanlong Ma, Rong Jin, Huiling Yuan, and Liang Sun
ECCV 2026 Spotlight verified in the official program, session Computational, Medical & Scientific Imaging; scientific claims use conference-hosted manuscript #7258. Publisher proceedings/version of record was not verified on 2026-09-12.
Official artifacts: ECCV Spotlight program · Conference manuscript · Author project (paper/program link)
Metadata checked 2026-09-12
#622025NeurIPSPeer reviewedL2–L3
Why Knowledge Distillation Works in Generative Models: A Minimal Working Explanation
Sungmin Cha and Kyunghyun Cho
Official artifacts: Official code
Metadata checked 2026-08-10
#632026Pattern RecognitionPeer reviewedL3
On learning denoisable student logits
Diqi Chen, Yang Li, Jiajun Liu, Brano Kusy, Jun Zhou, and Yongsheng Gao
Peer-reviewed article 113684 in Pattern Recognition 179; available online 10 April 2026 and assigned to the November 2026 issue.
Metadata checked 2026-09-02
#642026arXivPreprintL3
On-Policy Self-Distillation in Diffusion Models
Wei Zhou, Xiongwei Zhu, Lingdong Kong, Bo Chen, Lei Zhang, Yongyuan Liang, Xiaoxia Hou, Ye Tian, Xian Sun, Yingshuo Wang, Linfeng Li, Shengqiong Wu, Leigang Qu, Feng Li, Wei Liu, Julian McAuley, and Tat-Seng Chua
arXiv v1 technical report submitted 25 August 2026; reported reward gains rely on the chosen automated evaluators and two tested backbones.
Official artifacts: Official project · Official implementation
Metadata checked 2026-08-30
#652026arXivPreprintL3
Self-OPD: On-Policy Distillation for Flow Matching Models without Teacher
Shiyi Zhang, Mushui Liu, Yunze Tong, Wanggui He, Siyu Zou, Jinlong Liu, Yunlong Yu, Jian Song, Hao Jiang, Pipei Huang, and Bo Zheng
arXiv v1 preprint submitted 27 August 2026; teacher-free reward optimization remains benchmark- and reward-model-dependent.
Official artifacts: Official implementation
Metadata checked 2026-08-30