Skip to content
Computational Pathology Reading Roadmap

Generation, video, and multimodal agents

Visual generation, video-language modelling, and multimodal agents extend vision–language learning beyond a single image-to-text prediction. A generator emits pixels or visual tokens; a video model must preserve temporal evidence; an agent chooses observations and actions in a changing environment. These branches share language conditioning but require different objectives and safety boundaries.

Latest literature discovery audit: 2 October 2026. Existing paper reviews retain their individual source-check dates.

ECCV 2026 conference additions and metadata checked through 2026-09-12; the broader literature audit boundary above is unchanged.

Prerequisite path

Section titled “Prerequisite path”
  1. Complete generative modelling, autoregression, VAEs, and diffusion in the ML/deep-learning route.
  2. Read image–text representation and multimodal LLMs.
  3. For agents, understand reinforcement learning, partial observability, tool validation, and irreversible actions.

You are ready for research-level reading when you can distinguish a visual understanding token, a reconstruction token, a diffusion latent, a video observation, and an environment action.

Output, supervision, and evidence taxonomy

Section titled “Output, supervision, and evidence taxonomy”
Branch Output Supervision Evidence / state
Text-to-image/video pixels or latent visual sequence text–media pairs, denoising/flow/next-token loss prompt, optional reference/control inputs, training distribution
Image editing transformed image instruction–source–target triples or synthetic edits source image plus localized/global edit instruction
Unified understanding/generation text and images in either direction mixed causal, masked, diffusion, and preference objectives shared or separate representations with competing information needs
Video–text representation video/text embeddings paired clips and narration/captions sampled frames/clips; timing and audio may be lost
Video MLLM answer, caption, temporal segment, tool call video instructions, QA, temporal labels selected frames/tokens and optional subtitles/audio
GUI/web agent click, type, scroll, structured tool call demonstrations, screenshots/DOM, rewards current screen, accessibility/DOM metadata, interaction history
Embodied/VLA model discrete or continuous action trajectory robot demonstrations, language, vision, rewards camera/state history and embodiment-specific action space

Generation quality does not establish understanding; an agent’s successful final state does not establish that its intermediate plan was safe or visually grounded.

Method detail: visual generation and unification

Section titled “Method detail: visual generation and unification”

Representation and decoding choices

Section titled “Representation and decoding choices”
  • Discrete autoregression: quantize images and predict visual tokens sequentially; offers a language-like objective but can be slow and sensitive to tokenizer quality.
  • Diffusion/flow: iteratively transform noise into a visual latent; supports high-fidelity continuous detail but normally uses a separate text encoder/decoder.
  • Hybrid/unified: use autoregression for text and diffusion for continuous visual outputs, or one discrete vocabulary for both.
  • Early fusion: interleave text and image tokens in one causal sequence; simple interface, difficult optimization and long visual sequences.

The central unification question is not whether one checkpoint can run both tasks. It is whether shared training produces measurable, controlled synergy instead of negative transfer.

Order Primary paper and year Venue / status Read it for
1 VQ-VAE (2017) NeurIPS, peer reviewed discrete learned visual tokens and autoregressive priors
2 DALL·E (2021) ICML, peer reviewed autoregressive text-to-image generation with discrete image tokens
3 Latent Diffusion (2022) CVPR, peer reviewed diffusion in a learned visual latent and cross-attention conditioning
4 Imagen (2022) NeurIPS, peer reviewed scaling text encoders and cascaded diffusion
5 DiT (2023) ICCV, peer reviewed transformer backbones and compute scaling for diffusion
6 Chameleon (2024) arXiv preprint early-fusion, token-based mixed-modal generation
7 Transfusion (2025) ICLR, peer reviewed next-token text prediction plus diffusion for continuous modalities
8 Show-o (2025) ICLR, peer reviewed autoregressive text with discrete diffusion for images
9 Janus-Pro (2025) arXiv preprint decoupled visual pathways in a unified understanding/generation model
10 JanusFlow (2025) CVPR, peer reviewed autoregressive understanding coupled to rectified-flow visual generation
11 OneCAT (2026) CVPR, peer reviewed decoder-only autoregressive understanding, generation, and editing
12 RealUnify (2026) CVPR, peer reviewed controlled evaluation of cross-capability synergy
13 SenseNova-Vision (2026) arXiv preprint text, image, and mixed generation as a common interface for structured vision tasks

Diffusion design, conditioning, personalization, and editing

Section titled “Diffusion design, conditioning, personalization, and editing”

These papers separate improvements to the diffusion process from ways of supplying a source image, subject, spatial structure, or natural-language edit. Record whether a method changes the backbone, duplicates a frozen branch, fine-tunes per subject, or depends on synthetic instruction–image pairs.

Order Primary paper and year Venue / status Read it for
1 GLIDE (2022) ICML, peer reviewed text-guided diffusion, classifier-free guidance, and inpainting/editing
2 EDM (2022) NeurIPS, peer reviewed separating preconditioning, noise schedule, training, and sampler design
3 DreamBooth (2023) CVPR, peer reviewed subject-driven personalization and the prior-preservation trade-off
4 InstructPix2Pix (2023) CVPR, peer reviewed synthetic edit pairs and instruction-conditioned source-image editing
5 ControlNet (2023) ICCV, peer reviewed preserving a pretrained diffusion backbone while adding spatial controls

Text-to-video generation

Section titled “Text-to-video generation”

Generation must be evaluated for prompt binding and temporal consistency, not only per-frame realism. Record visual tokenizer, causal or bidirectional factorization, duration, frame rate, conditioning modalities, sampling steps, and whether reported efficiency holds at matched resolution and clip length.

Order Primary paper and year Venue / status Read it for
1 Phenaki (2023) ICLR, peer reviewed variable-length causal video tokens and story-like prompt sequences
2 VideoPoet (2024) ICML, peer reviewed one token-based language model across video, image, audio, and editing tasks
3 M4V (2026) CVPR, peer reviewed state-space sequence modelling for efficient text-conditioned video generation

Video–language learning

Section titled “Video–language learning”

Video adds a sampling policy before the model. Duration, frames per second, clip boundaries, audio/subtitles, and temporal ordering are part of the input contract.

Representation and retrieval lineage

Section titled “Representation and retrieval lineage”
Order Primary paper and year Venue / status Read it for
1 VideoBERT (2019) ICCV, peer reviewed discrete visual tokens and bidirectional video–language pretraining
2 HowTo100M (2019) ICCV, peer reviewed large-scale weak narration alignment and temporal noise
3 MIL-NCE (2020) CVPR, peer reviewed multiple-instance contrast for uncertain narration timing
4 Frozen in Time (2021) ICCV, peer reviewed joint image/video–text retrieval and temporal attention
5 VideoCLIP (2021) EMNLP, peer reviewed contrastive video/text pretraining with temporally overlapping positives
6 InternVideo (2022) arXiv preprint combining masked video modelling and video–text contrast

Video MLLMs and long-context evidence

Section titled “Video MLLMs and long-context evidence”
Order Primary paper and year Venue / status Read it for
1 Video-LLaMA (2023) EMNLP System Demonstrations, peer reviewed audio/visual encoders connected to a language model
2 VideoChatGPT (2024) ACL, peer reviewed instruction tuning and quantitative video-conversation evaluation
3 InternVideo2.5 (2025) arXiv preprint long and rich video context with memory and compression
4 Apollo (2025) CVPR, peer reviewed controlled study of model, data, duration, and token choices
5 Video-XL (2025) CVPR, peer reviewed visual summarization tokens and hour-scale context
6 VITAL (2026) CVPR, peer reviewed tool-controlled frame acquisition during long-video reasoning
7 Chain-of-Frames (2026) CVPR, peer reviewed reasoning traces with explicit relevant-frame references
8 VideoAuto-R1 (2026) CVPR, peer reviewed deciding when chain-of-thought is worth its computation
9 LensWalk (2026) CVPR, peer reviewed a reason–plan–observe loop for active temporal evidence gathering

Multimodal agents and action

Section titled “Multimodal agents and action”

An agent operates in a loop:

observation → visual grounding → state/history → plan → action/tool
            → environment transition → verification or recovery

The environment, observation API, action grammar, and reset policy are part of the model. HTML/DOM or robot-state access can supply information absent from screenshots; report it rather than calling the system vision-only.

Order Primary paper and year Venue / status Read it for
1 SayCan (2023; CoRL 2022) PMLR version of record, peer reviewed combining language-model plans with learned affordance values
2 RT-1 (2023) Robotics: Science and Systems, peer reviewed tokenizing real-world robot observations and actions at scale
3 PaLM-E (2023) ICML, peer reviewed continuous sensor/vision inputs inside an embodied language model
4 RT-2 (2023) CoRL, peer reviewed representing robot actions as text tokens for vision–language–action transfer
5 VisualWebArena (2024) ACL, peer reviewed realistic visually grounded web-agent tasks
6 OSWorld (2024) NeurIPS Datasets and Benchmarks, peer reviewed open-ended computer tasks across real applications
7 Magma (2025) CVPR, peer reviewed shared image, video, GUI, and robotics action-grounding training
8 ShowUI (2025) CVPR, peer reviewed screenshot grounding, UI token selection, and action streaming
9 OpenVLA (2025; CoRL 2024) PMLR version of record, peer reviewed an open 7B VLA and efficient adaptation across robot tasks
10 RiOSWorld (2025) NeurIPS, peer reviewed user-originated and environmental risks during real computer manipulation
11 ARM-Thinker (2026) CVPR, peer reviewed reward-model judgments grounded through crop and retrieval tools
12 ReFAct (2026) CVPR, peer reviewed adaptive visual/context focus in multimodal web-search agents
13 SecureWebArena (2026) Findings of ACL, peer reviewed adversarial web trajectories decomposed across reasoning, behavior, and outcome

Qwen2-VL and Qwen2.5-VL are also indexed on this route because their dynamic-resolution perception, grounding, and video interfaces can enable agents. Treat them as perception backbones, not evidence that an end-to-end policy can act safely.

Evaluation contract

Section titled “Evaluation contract”
Branch Core metrics Required controls
Image generation human prompt fidelity, object/attribute/relation accuracy, diversity memorization, demographic bias, unsafe prompts, equal sampling budget
Unified model understanding and generation separately plus synergy tasks matched capacity/data, negative transfer, representation interventions
Video retrieval Recall@K, median rank temporal shuffle, single-frame, subtitle/audio removal
Video QA/grounding accuracy, temporal IoU, evidence-frame recall fixed frame/token budget, duration bins, answerability
GUI/web agent task success and step success invalid actions, recovery, DOM versus screenshot access, environment version
Embodied/VLA success, collision/safety, intervention, trajectory cost embodiment shift, unseen objects/scenes, safe-stop behavior
Tool-using reasoner final correctness and tool validity unnecessary calls, bad tool outputs, evidence trace, latency/token cost

Controlled experiment

Section titled “Controlled experiment”

Run three linked studies:

  1. Unified generation: compare separate understanding/generation checkpoints with a unified checkpoint under matched parameter and data budgets. Test ordinary tasks and cross-capability tasks where an image must be understood before it is edited.
  2. Video evidence: fix a maximum visual-token budget and compare uniform frames, learned compression, and agent-selected frames. Add temporal reversal, shuffling, subtitle removal, and a single-frame baseline.
  3. Agent loop: replay a version-pinned environment from identical initial states. Compare screenshot-only, screenshot+structured metadata, and active crop/tool access. Record success, invalid actions, recovery, interventions, latency, tokens, and tool calls.

Do not compare success rates from different environment versions or action APIs.

Failure modes

Section titled “Failure modes”
  • FID or CLIP score rewards superficial fidelity and misses prompt contradictions.
  • A generator memorizes or reconstructs training images.
  • Unified training improves one task only because it receives more data or parameters.
  • Long-video results use more frames/tokens than baselines.
  • Subtitles answer the question without temporal visual understanding.
  • Uniform frame sampling misses short events; aggressive compression misses small text.
  • Agent benchmarks change websites, apps, or simulators between runs.
  • Structured metadata supplies the target hidden from screenshots.
  • Final success hides unsafe intermediate actions or excessive human correction.
  • Tool outputs are accepted without validation or provenance.
  • A model plans fluently after an incorrect visual observation.
  • Embodied policies are transferred across robots without action-space calibration.

Self-check and exit task

Section titled “Self-check and exit task”
  1. Why do understanding and generation favor different visual representations?
  2. What controlled result would demonstrate useful unification?
  3. Which video input choices must be reported before architecture?
  4. How can a single-frame or subtitle-only model expose benchmark shortcuts?
  5. What distinguishes an MLLM that recommends actions from a VLA policy?
  6. Why must final task success be decomposed into perception, planning, execution, and recovery?

Exit task: select one unified model, one video model, or one agent. Reconstruct its data/objective/environment contract, then create one equal-budget baseline and three interventions that target its claimed visual or temporal evidence.

Bridge to computational pathology

Section titled “Bridge to computational pathology”

Image generation in pathology requires morphology- and stain-aware fidelity checks, not natural-image preference scores. Video-style active observation is relevant to WSI navigation because an agent may choose magnification, region, or ancillary evidence over time. However, slide actions must be non-destructive and logged, and diagnostic outputs need expert protocols, abstention, and patient-level evaluation. Use the WSI engineering and pathology VLM chapters before transferring a natural-image generator or agent.

Readable prompt ensembles, local concepts and selective adaptation

TPB reweights mistakes across discrete prompt classifiers, CFM exposes spatially grounded sparse concepts, and Robotic Steering selects action-predictive attention heads. Compare what is selected or trained, how frozen features are reused, and whether the final intervention preserves unrelated capabilities.

Primary paper · year · evidenceCore idea and why to read itOfficial resources
Mechanistic Finetuning of Vision-Language-Action Models via Few-Shot DemonstrationsChancharik Mitra, Yusen Luo, Raj Saravanan, Dantong Niu, Anirudh Pai, Jesse Thomason, Trevor Darrell, Abrar Anwar, Deva Ramanan, and Roei Herzig2026 · ECCV · acceptedECCV 2026 Spotlight verified in the official Vision-Language Models & Visual Foundation Models session; manuscript #13378 checked on 2026-09-12. Publisher version of record was not independently verified.Robotic Steering chooses which VLA attention heads to adapt using a few demonstrations. Head activations are scored by how well cross-trajectory nearest-neighbor retrieval predicts demonstrated actions. Low-rank updates then target selected query and output projections. The central contribution is data-driven adaptation location, not a new distillation loss or an inference-time steering vector.ECCV Spotlight program · Conference manuscript · Official project or implementation

Reading question: Is the adaptation unit a prompt bank, a concept dictionary, an attention head, or another trainable branch?

Evidence boundary: Frozen backbones do not imply label-free or zero-cost adaptation. Probe predictability is not causal proof, and Robotic Steering also trains FFN LoRA and its action head.

Counterexamples for spatial, semantic and action grounding

Use matched distance counterexamples, cross-category part correspondence, action-absent video clips and background-consistent concealment to test different kinds of visual evidence. Preserve each study's denominator: answer probability, correspondence success, absence/presence accuracy and newly introduced ungrounded nouns are not interchangeable.

Primary paper · year · evidenceCore idea and why to read itOfficial resources
Learning to Deny: Action Denial in Multimodal Large Language ModelsRaiyaan Abdullah, Shehreen Azad, and Yogesh Singh Rawat2026 · ECCV · acceptedECCV 2026 Spotlight verified in the official Vision-Language Models & Visual Foundation Models session; manuscript #3692 checked on 2026-09-12. Publisher version of record was not independently verified.Learning to Deny introduces UCF101-AD to test whether video-language models reject an action whose defining motion never occurs despite plausible context. Its CausalAct framework organizes persons, objects, relations, interactions and motion into an explicit verification order. The paper combines a denial benchmark with a structured-prompt and fine-tuning baseline.ECCV Spotlight program · Conference manuscript · Official project or implementation

Reading question: Does an apparent improvement survive counterexamples that preserve superficial cues while changing the correct spatial, semantic or temporal conclusion?

Evidence boundary: A small bias gap can coexist with near-chance performance. Action denial is not safety refusal, detector-based concealment is not a privacy guarantee, and benchmark correlations are not universal causal explanations.

Reasoning budgets, executable outputs and scientific edits

Test-time scaling studies how extra reasoning changes perception and mathematics; EmbedCopilot checks whether plausible embedded-system answers actually execute; GRADE requires domain-correct and readable image edits. Together they motivate measuring compute, implementation success and constraint satisfaction rather than judging prose alone.

Primary paper · year · evidenceCore idea and why to read itOfficial resources
EmbedCopilot-Bench: Evaluating Vision-Language Models for Hardware-Aware Embedded System DevelopmentDongsheng Yuan, Yimo Deng, and Huangxun Chen2026 · ECCV · acceptedECCV 2026 Spotlight verified in the official Vision-Language Models & Visual Foundation Models session; manuscript #9127 checked on 2026-09-12. Publisher version of record was not independently verified. The PDF uses EmbedCopilot-Bench; the program shortens this to EmbedCopilot.EmbedCopilot-Bench evaluates whether a vision-language assistant can turn hardware and software screenshots into useful embedded-development steps and functioning code. It combines rubric-based response quality with execution success, explicitly separating exact pin-grounded operation from behavior that works only after pin correction. Fluent instructions alone are insufficient.ECCV Spotlight program · Conference manuscript · Official project or implementation
GRADE: Benchmarking Discipline-Informed Reasoning in Image EditingMingxin Liu, Ziqian Fan, Zhaokai Wang, Leyao Gu, Zirun Zhu, Yiguo He, Yuchen Yang, Changyao Tian, Xiangyu Zhao, Ning Liao, Shaofeng Zhang, Qibing Ren, Zhihang Zhong, Xuanhe Zhou, Junchi Yan, and Xue Yang2026 · ECCV · acceptedECCV 2026 Spotlight verified in the official Vision-Language Models & Visual Foundation Models session; manuscript #2706 checked on 2026-09-12. Publisher version of record was not independently verified.GRADE evaluates whether an image editor can use disciplinary knowledge to make a scientifically correct visual change, preserve relevant context, and keep the result logically readable. Its 520 image–instruction–reference triplets span ten academic disciplines. The central distinction is between drawing a plausible image and rendering the right answer to a domain-specific problem.ECCV Spotlight program · Conference manuscript · Official project or implementation

Reading question: What external test can falsify a persuasive explanation: a perception answer, a running circuit, or a scientifically valid image?

Evidence boundary: More tokens can reduce perceptual accuracy. Judge scores and physical execution use different protocols; GRADE strict accuracy is a conjunction of three maxima, not its mean reasoning score.

Frequency-domain visual token budgets

Fourier Compressor uses fixed DCT truncation on feature grids, followed by VLM adaptation. Distinguish parameter-free compression from training-free deployment, and average retention from fine-detail fidelity.

Primary paper · year · evidenceCore idea and why to read itOfficial resources
Fourier Compressor: Frequency-Domain Visual Token Compression for Vision-Language ModelsHuanyu Wang, Jushi Kai, Haoli Bai, Lu Hou, Bo Jiang, Ziwei He, Zhouhan Lin2026 · ECCV · acceptedECCV 2026 spotlight verified from the official linked program event. Review uses the conference-hosted manuscript checked 2026-09-12; a separate Springer version of record was not verified. Parameter-free compression is NOT training-free: Section 5.1 and Table 2 require adaptation. The 31.2% number concerns reduced time to first token, not a measured 31.2% increase in autoregressive decoding throughput.Fourier Compressor removes visual tokens by shrinking their spatial frequency representation. It applies a two-dimensional type-II DCT independently across feature channels, keeps a low-frequency rectangle, and applies an inverse DCT on the smaller grid. The compressor adds no learned parameters, but the surrounding VLM is trained to accommodate the altered features.Official ECCV spotlight record · Official code

Reading question: Which frequencies and tasks lose evidence at a matched training and token budget?

Evidence boundary: TextVQA and video subtasks can regress despite strong average retention; TTFT is not decoding throughput.

Low-resolution student inference versus resolution-cascaded generation

GAP-Track removes teacher and residual reconstruction branches at inference. RMD instead retains high-resolution generative refinement. Both exploit cross-resolution supervision but solve different deployment problems.

Primary paper · year · evidenceCore idea and why to read itOfficial resources
Cross-Resolution Distribution Matching for Diffusion DistillationFeiyang Chen, Hongpeng Pan, Haonan Xu, Xinyu Duan, Zhefeng Wang, and Yang Yang2026 · ECCV · acceptedECCV 2026 poster verified from the official linked program event. Review uses the conference-hosted manuscript checked 2026-09-12; a separate Springer version of record was not verified. RMD retains high-resolution refinement at inference; it is not low-resolution-only recognition. Reported speedups depend on the single-A100 baseline, CFG and NFE. The conference PDF author order differs from the arXiv record and is preserved here.RMD distills a diffusion generator into a few-step trajectory that begins at low resolution and ends at high resolution. It aligns resolution-specific noise states using logSNR, projects student distributions into the teacher’s resolution, and uses distribution matching plus noise reinjection to train the cascade. The output remains a high-resolution generated image or video.Official ECCV poster record

Reading question: Is high resolution absent at deployment, or only used for fewer steps? What is the full-resolution reference?

Evidence boundary: Synthetic downsampling does not reproduce all optical changes between microscope magnifications; lower-resolution gains need not equal full-resolution performance.

Perceptual experts, geometry and explicit grounding tokens

Continuous expert-supervised visual tokens, geometry-aware fusion and coarse-to-fine grounding alter how a language model accesses visual evidence. Compare the representation interface, auxiliary supervision and deployed computation before comparing answer scores.

Primary paper · year · evidenceCore idea and why to read itOfficial resources
MolmoPoint: Better Pointing for VLMs with Grounding TokensChristopher Clark, Yue Yang, Jae Sung Park, Zixian Ma, Jieyu Zhang, Rohun Tripathi, Mohammadreza Salehi, Sangho Lee, and Ranjay Krishna2026 · ECCV · acceptedECCV 2026 Spotlight verified in the official program (event 6028); scientific claims use the conference-hosted manuscript #8311. Publisher proceedings/version of record was not verified on 2026-09-12. The program spells the name Molmo-Point; the conference PDF uses MolmoPoint.MolmoPoint makes a VLM point by selecting its own visual tokens, then refining the selected location. Three grounding tokens replace a coordinate string: choose a coarse patch, select a finer ViT subpatch, and choose a position inside that subpatch. The mechanism ties spatial output to visual evidence and reduces the need to learn a global textual coordinate system.ECCV program · Conference manuscript · Author project

Reading question: Does a claimed reasoning gain arise from more visual evidence, better alignment, a changed output space or a larger inference budget?

Evidence boundary: Expert distillation is not proof of faithful reasoning; stronger pointing or spatial QA need not improve every tracking metric.

Long-video memory, scripts and entity continuity

Retrieval-based episodic memory, audio-visual script generation and entity-centric evaluation answer different long-video questions. Track what is stored, how evidence is retrieved, and whether identities and events remain consistent across time.

Primary paper · year · evidenceCore idea and why to read itOfficial resources
Keep It Simple: Multi-Key Episodic Memory Retrieval for Ultra-Long Video UnderstandingYeeun Choi, Youngbeom Yoo, Joon-Young Lee, Hyolim Kang, and Seon Joo Kim2026 · ECCV · acceptedECCV 2026 Oral verified in the official program (event 6047); scientific claims use the conference-hosted manuscript #10831. Publisher proceedings/version of record was not verified on 2026-09-12.MERIT uses multi-key episodic memory for ultra-long video QA: preserve short, detailed episodes when building memory, then compose query-specific temporal context only after the question is known. Each episode has event, dialogue, object, and summary keys; retrieval uses its strongest matching key, followed by local temporal expansion and question-aware filtering.ECCV program · Conference manuscript · Author project · Code
OmniScript: Towards Audio-Visual Script Generation for Long-Form Cinematic VideoJunfu Pu, Yuxin Chen, Teng Wang, and Ying Shan2026 · ECCV · acceptedECCV 2026 Oral verified in the official program (event 6045); scientific claims use the conference-hosted manuscript #9919. Publisher proceedings/version of record was not verified on 2026-09-12.OmniScript converts cinematic audio and video into a hierarchical scene-event script. The model first reasons about plot and character relationships, then emits time-grounded fields for actions, dialogue, expressions, and audio cues. Its research contribution couples an audio-visual architecture, structured training data, and an evaluation protocol that separately checks what happened and when.ECCV program · Conference manuscript · Author project · Code
NarrativeTrack: Evaluating Entity-Centric Reasoning for Narrative UnderstandingHyeonjeong Ha, Jinjin Ge, Bo Feng, Kaixin Ma, and Gargi Chakraborty2026 · ECCV · acceptedECCV 2026 Spotlight verified in the official program (event 6177); scientific claims use the conference-hosted manuscript #10972. Publisher proceedings/version of record was not verified on 2026-09-12. Earlier arXiv versions used the subtitle Evaluating Video Language Models Beyond the Frame; this is the same canonical work.NarrativeTrack evaluates whether a video-language model keeps track of the same person through time, changing attributes, and visually confusable characters. Its Compositional Reasoning Progression moves from entity existence to state changes and then identity ambiguity. This makes narrative understanding a set of testable temporal-perceptual operations rather than one broad caption-quality score.ECCV program · Conference manuscript · Author project · Dataset

Reading question: Can the system recover the right episode and preserve who did what, when, and in which scene?

Evidence boundary: Compressed memory can omit decisive events; fluent scripts and pooled benchmark scores can conceal identity or temporal failures.

Causal audiovisual generation and variable-step distillation

OmniForcing transfers a bidirectional audio-video generator to causal streaming; AnyFlow learns on-policy flow maps supporting different sampling budgets. Preserve training-stage, self-rollout, synchronization, step-count and hardware qualifiers.

Primary paper · year · evidenceCore idea and why to read itOfficial resources
OmniForcing: Unleashing Real-time Joint Audio-Visual GenerationYaofeng Su, Yuming Li, Zeyue Xue, Jie Huang, Siming Fu, Haoran Li, Haoyang Huang, and Nan Duan2026 · ECCV · acceptedECCV 2026 Oral verified in the official program; scientific claims use the conference-hosted manuscript #9084. Publisher proceedings/version of record was not verified on 2026-09-12. Author list follows the conference manuscript and program; the current repository citation lists two additional authors. Review separates the paper's three-stage method from the later simplified two-stage code release.OmniForcing converts the 19B LTX-2 bidirectional audio–video generator into a streaming model through causal distillation. Its defining problem is not simply fewer diffusion evaluations: video and audio have different latent time scales, yet both must be generated causally and remain synchronized. The paper combines physically aligned one-second blocks, stabilized causal attention and joint self-forcing. It reports a 0.7-second first chunk and 25 FPS on one H200, while retaining explicit quality tradeoffs against the slower teacher.ECCV Oral program · Conference manuscript · Official implementation · arXiv manuscript · Author project · Model checkpoint

Reading question: What teacher information disappears at inference, and does quality remain stable as the generated history or step budget changes?

Evidence boundary: Real-time and any-step claims depend on resolution, duration, hardware and guidance; source-code revisions may change the training recipe.

Complete maintained paper route

Section titled “Complete maintained paper route”

72 canonical readings · 2017–2026· route reviewed through October 2, 2026.

This is the complete structured list for this chapter, ordered by an explicit prerequisite-aware teaching sequence. The explanatory ladder above highlights the shortest conceptual route; this list also includes important branches, benchmarks, and current endpoints. It is generated from the same deduplicated catalog used by the AI background registry and My Library, so a future catalog update appears here automatically.

Show all 72 readings in maintained order

  13. #132026ECCVAccepted · proceedings pendingL3

    GRADE: Benchmarking Discipline-Informed Reasoning in Image Editing

    Mingxin Liu, Ziqian Fan, Zhaokai Wang, Leyao Gu, Zirun Zhu, Yiguo He, Yuchen Yang, Changyao Tian, Xiangyu Zhao, Ning Liao, Shaofeng Zhang, Qibing Ren, Zhihang Zhong, Xuanhe Zhou, Junchi Yan, and Xue Yang

    ECCV 2026 Spotlight verified in the official Vision-Language Models & Visual Foundation Models session; manuscript #2706 checked on 2026-09-12. Publisher version of record was not independently verified.

    Official artifacts: ECCV Spotlight program · Conference manuscript · Official project or implementation

    Metadata checked

  33. #332026ECCVAccepted · proceedings pendingL3

    Cross-Resolution Distribution Matching for Diffusion Distillation

    Feiyang Chen, Hongpeng Pan, Haonan Xu, Xinyu Duan, Zhefeng Wang, and Yang Yang

    ECCV 2026 poster verified from the official linked program event. Review uses the conference-hosted manuscript checked 2026-09-12; a separate Springer version of record was not verified. RMD retains high-resolution refinement at inference; it is not low-resolution-only recognition. Reported speedups depend on the single-A100 baseline, CFG and NFE. The conference PDF author order differs from the arXiv record and is preserved here.

    Official artifacts: Official ECCV poster record

    Metadata checked

  34. #342026ECCVAccepted · proceedings pendingL3

    OmniForcing: Unleashing Real-time Joint Audio-Visual Generation

    Yaofeng Su, Yuming Li, Zeyue Xue, Jie Huang, Siming Fu, Haoran Li, Haoyang Huang, and Nan Duan

    ECCV 2026 Oral verified in the official program; scientific claims use the conference-hosted manuscript #9084. Publisher proceedings/version of record was not verified on 2026-09-12. Author list follows the conference manuscript and program; the current repository citation lists two additional authors. Review separates the paper's three-stage method from the later simplified two-stage code release.

    Official artifacts: ECCV Oral program · Conference manuscript · Official implementation · arXiv manuscript · Author project · Model checkpoint

    Metadata checked

  40. #402026ECCVAccepted · proceedings pendingL3

    Keep It Simple: Multi-Key Episodic Memory Retrieval for Ultra-Long Video Understanding

    Yeeun Choi, Youngbeom Yoo, Joon-Young Lee, Hyolim Kang, and Seon Joo Kim

    ECCV 2026 Oral verified in the official program (event 6047); scientific claims use the conference-hosted manuscript #10831. Publisher proceedings/version of record was not verified on 2026-09-12.

    Official artifacts: ECCV program · Conference manuscript · Author project · Code

    Metadata checked

  42. #422026ECCVAccepted · proceedings pendingL3

    NarrativeTrack: Evaluating Entity-Centric Reasoning for Narrative Understanding

    Hyeonjeong Ha, Jinjin Ge, Bo Feng, Kaixin Ma, and Gargi Chakraborty

    ECCV 2026 Spotlight verified in the official program (event 6177); scientific claims use the conference-hosted manuscript #10972. Publisher proceedings/version of record was not verified on 2026-09-12. Earlier arXiv versions used the subtitle Evaluating Video Language Models Beyond the Frame; this is the same canonical work.

    Official artifacts: ECCV program · Conference manuscript · Author project · Dataset

    Metadata checked

  58. #582026ECCVAccepted · proceedings pendingL3

    Fourier Compressor: Frequency-Domain Visual Token Compression for Vision-Language Models

    Huanyu Wang, Jushi Kai, Haoli Bai, Lu Hou, Bo Jiang, Ziwei He, Zhouhan Lin

    ECCV 2026 spotlight verified from the official linked program event. Review uses the conference-hosted manuscript checked 2026-09-12; a separate Springer version of record was not verified. Parameter-free compression is NOT training-free: Section 5.1 and Table 2 require adaptation. The 31.2% number concerns reduced time to first token, not a measured 31.2% increase in autoregressive decoding throughput.

    Official artifacts: Official ECCV spotlight record · Official code

    Metadata checked

  59. #592026ECCVAccepted · proceedings pendingL3

    MolmoPoint: Better Pointing for VLMs with Grounding Tokens

    Christopher Clark, Yue Yang, Jae Sung Park, Zixian Ma, Jieyu Zhang, Rohun Tripathi, Mohammadreza Salehi, Sangho Lee, and Ranjay Krishna

    ECCV 2026 Spotlight verified in the official program (event 6028); scientific claims use the conference-hosted manuscript #8311. Publisher proceedings/version of record was not verified on 2026-09-12. The program spells the name Molmo-Point; the conference PDF uses MolmoPoint.

    Official artifacts: ECCV program · Conference manuscript · Author project

    Metadata checked

  61. #612026ECCVAccepted · proceedings pendingL3

    Mechanistic Finetuning of Vision-Language-Action Models via Few-Shot Demonstrations

    Chancharik Mitra, Yusen Luo, Raj Saravanan, Dantong Niu, Anirudh Pai, Jesse Thomason, Trevor Darrell, Abrar Anwar, Deva Ramanan, and Roei Herzig

    ECCV 2026 Spotlight verified in the official Vision-Language Models & Visual Foundation Models session; manuscript #13378 checked on 2026-09-12. Publisher version of record was not independently verified.

    Official artifacts: ECCV Spotlight program · Conference manuscript · Official project or implementation

    Metadata checked

  65. #652026ECCVAccepted · proceedings pendingL3

    Learning to Deny: Action Denial in Multimodal Large Language Models

    Raiyaan Abdullah, Shehreen Azad, and Yogesh Singh Rawat

    ECCV 2026 Spotlight verified in the official Vision-Language Models & Visual Foundation Models session; manuscript #3692 checked on 2026-09-12. Publisher version of record was not independently verified.

    Official artifacts: ECCV Spotlight program · Conference manuscript · Official project or implementation

    Metadata checked

  69. #692026ECCVAccepted · proceedings pendingL3

    EmbedCopilot-Bench: Evaluating Vision-Language Models for Hardware-Aware Embedded System Development

    Dongsheng Yuan, Yimo Deng, and Huangxun Chen

    ECCV 2026 Spotlight verified in the official Vision-Language Models & Visual Foundation Models session; manuscript #9127 checked on 2026-09-12. Publisher version of record was not independently verified. The PDF uses EmbedCopilot-Bench; the program shortens this to EmbedCopilot.

    Official artifacts: ECCV Spotlight program · Conference manuscript · Official project or implementation

    Metadata checked

  71. #712026arXivPreprintL2–L3

    LAION-BVD: A 10-Million-Hour Open Video Dataset for Multimodal Pre-training

    Andreas Hochlehnert, Marianna Nezhurina, Mehdi Cherti, Andrej Radonjic, Thaddäus Wiedemer, Christoph Schuhmann, Romain Beaumont, Wieland Brendel, Bernhard Schölkopf, A. Sophia Koepke, Jenia Jitsev, and Matthias Bethge

    arXiv preprint first submitted 25 August 2026; not peer reviewed as of the audit date.

    Official artifacts: Official project · Official data tooling · Official URL dataset

    Metadata checked

  72. #722026arXivPreprintL2–L3

    VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning

    Junxiang Xu, Ruisi Wang, Fanyi Pu, Maijunxian Wang, Ran Ji, Tongxi Zhou, Chenyang Gu, Jing Zuo, Hongcan Xiao, Yimeng Geng, Wanqi Yin, Wei Chen, Oscar Qian, Zhengan Yan, Ziqi Huang, Haiwen Diao, Liang Pan, Bo Li, Xiangyu Fan, Dezhi Luo, Fengyuan Yu, Zehong Zhao, Qingying Gao, Tinghui Zhu, Yilan Zhang, Jingqi Tong, Pinyuan Feng, Zhengze Jiang, Letian Wang, Ziyu Guo, Renrui Zhang, Jieneng Chen, Sonia Joseph, Constantin Venhoff, Saman Motamed, Mengyue Yang, Chandra Sripada, Alan Yuille, Philip Torr, Lvmin Zhang, Vikash Kumar, Daniel Khashabi, Nikolaus Kriegeskorte, Raphaël Millière, Vincent C. Müller, Anyi Rao, Quan Wang, Ziwei Liu, Dahua Lin, Lei Yang, Hokin Deng, and Zhongang Cai

    arXiv preprint first submitted 26 August 2026; not peer reviewed as of the audit date.

    Official artifacts: Official project · Official code and benchmark

    Metadata checked