Vision–language representation
Document models align visual regions, tokens, and generated sequences.
Document intelligence
Documents bind language to layout and graphics. This chapter separates recognition, grounding, reasoning, schema decoding, and exact structural validity.
19 maintained papers · literature reviewed through 2 October 2026
Documents bind words to two-dimensional layout, reading order, tables, figures, charts, and schemas. OCR pipelines, OCR-free encoders, screenshot parsing, and multimodal LLMs differ in where recognition errors enter and how structure is represented. Evaluation must isolate text reading, layout grounding, reasoning, and exact serialization.
Prerequisite contract
Structured NLP outputs
Entities, relations, trees, and serialized schemas need explicit matching.
How the field developed
2019–2020
Text-aware VQA
OCR-VQA, TextVQA, and M4C expose reading and copying text from natural scenes and images.
Key papers in this section
Order Primary paper · authors · year · venue/status Core idea and why to read it Official resources 01.01 OCR-VQA: Visual Question Answering by Reading Text in Images OCR-VQA–200K turns 207,572 book-cover images and their catalog metadata into 1,002,146 question-answer pairs about titles, authors, genres, years, and editions. Questions begin from templates, receive human paraphrases, and are paired with metadata-derived answers; images are randomly split before the accompanying OCR-plus-VQA baseline is trained. The central contribution is a large reading-oriented benchmark, not the claim that its baseline solves open-ended text reasoning. Memory line: cover plus metadata, template plus paraphrase, image split, OCR bottleneck. — 01.02 Towards VQA Models That Can Read TextVQA is an open-ended benchmark for questions whose answer requires reading text in an everyday photograph and often reasoning about that text. It contains 45,336 questions on 28,408 Open Images v3 images, with ten human answers per question and image-disjoint train, validation, and test partitions. The accompanying LoRRA baseline adds OCR-token attention and a dynamic copy space to a conventional VQA system, but model architecture is evidence about the benchmark—not the benchmark itself. Memory line: look at objects, read scene text, reason, then generate or copy. — 01.03 Iterative Answer Prediction With Pointer-Augmented Multimodal Transformers for TextVQA M4C answers questions about text inside an image by jointly contextualizing question tokens, detected visual objects, and OCR tokens, then generating a multiword answer with a pointer-augmented autoregressive decoder. At every step it can emit a frequent word from a fixed 5,000-word answer vocabulary or copy one of the image's OCR tokens. This changes TextVQA from one-shot selection of one answer or one OCR token into conditional sequence generation. Memory line: embed three modalities, self-attend once, copy or generate repeatedly. — Work: Trace OCR candidates, spatial features, copying, vocabulary decoding, and answer normalization.
Exit evidence: Separate OCR recall, grounding, reasoning, and answer parsing errors.
2020–2022
Layout-aware document pretraining
LayoutLM adds two-dimensional positions to OCR tokens; LayoutLMv2 adds page pixels; DocFormer instead uses shared spatial encodings and multimodal self-attention; LayoutLMv3 unifies masked text and image modelling with word–patch alignment.
Key papers in this section
Order Primary paper · authors · year · venue/status Core idea and why to read it Official resources 02.01 LayoutLM: Pre-training of Text and Layout for Document Image Understanding LayoutLM extends BERT-style document pretraining by representing each OCR word with both its text embedding and its normalized two-dimensional bounding-box coordinates. Masked visual-language modeling then learns words in the context of neighboring language and page geometry; an optional Faster R-CNN image feature adds appearance during fine-tuning. The central insight is that document meaning depends on where text appears, not only on token order. Official implementation and models 02.02 LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding LayoutLMv2 jointly encodes OCR text, two-dimensional bounding boxes, and page pixels. Text tokens receive ordinary sequence and 2D layout embeddings; a CNN-FPN converts the document image into visual tokens; a multimodal Transformer mixes both streams with spatial-aware attention. Pretraining masks text, tests whether text lines align with image regions, and classifies whether an image and text page match, forcing the model to learn content, geometry, and appearance together. Official implementation 02.03 DocFormer: End-to-End Transformer for Document Understanding DocFormer is an encoder-only document Transformer that keeps visual, language, and spatial streams distinct while making them interact at every layer. ResNet page features and OCR word embeddings each attend with modality-specific projections; shared spatial-attention weights tie their page geometry, and the two attended streams are added into a common hidden state. Multimodal language, image-reconstruction, and image–text matching objectives force the representation to use more than text alone. — 02.04 LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking LayoutLMv3 simplifies multimodal document pretraining by representing page images as linearly projected patches and training text and image with parallel masking objectives in one Transformer. Masked language modeling recovers OCR words, masked image modeling predicts discrete visual tokens, and word–patch alignment asks whether the image patch corresponding to a word was masked. The result is one backbone intended for both text-centric extraction and image-centric layout analysis. Official implementation and models Work: Hold OCR, pages, downstream heads, and splits fixed while ablating text, pixels, two-dimensional coordinates, attention interface, and pretraining losses.
Exit evidence: Attribute each document gain to text, layout, pixels, or objective rather than to a model name.
2021–2022
Document, table, and chart benchmarks
DocVQA and ChartQA test question answering under distinct evidence contracts; PubTables-1M supplies canonicalized table detection, structure-recognition, and functional-analysis labels at scale.
Key papers in this section
Order Primary paper · authors · year · venue/status Core idea and why to read it Official resources 03.01 DocVQA: A Dataset for VQA on Document Images DocVQA reframes document understanding as purpose-driven question answering: given a scanned document image and a natural-language question, return a text span found on that page. Its 50,000 manually annotated questions cover 12,767 page images sampled from 6,071 historical industry documents. The benchmark sits between visual question answering and extractive reading comprehension because success requires OCR, layout and structure, question-conditioned retrieval, and exact answer extraction rather than generic document classification. — 03.02 PubTables-1M: Towards Comprehensive Table Extraction From Unstructured Documents PubTables-1M is a large scientific-document resource for table detection, table structure recognition, and functional analysis. Its defining contribution is not only scale—about 948,000 tables—but canonicalization: logically equivalent yet differently authored markup is transformed into one consistent grid representation, with cell content, topology, locations, row/column structure, headers, projected row headers, and spanning cells. The resource makes annotation consistency measurable rather than assuming source markup is ground truth. Official dataset, code, and Table Transformer models 03.03 ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning ChartQA is a real-world chart-question benchmark built to require both visual reference resolution and logical or arithmetic reasoning. Its human track contains 9,608 questions over 4,804 charts collected from Statista, Pew Research, Our World in Data, and OECD; a second track adds 23,111 machine-generated questions over 17,141 Statista charts. The paper evaluates both an oracle setting with the underlying data table and a realistic pipeline that must extract that table from the chart image, making perception error and reasoning error separately visible. — Work: Separate OCR, layout, table topology, graphical marks, arithmetic, and answer normalization without changing evaluation data.
Exit evidence: Report structural validity and evidence grounding separately from final string accuracy.
2022–2026
OCR-free parsing, visual retrieval, and harder charts
Donut and Pix2Struct decode structures from pixels; ColPali retrieves page images with token-level late interaction; ChartQAPro and PolyChartQA broaden chart complexity and evaluation.
Key papers in this section
Order Primary paper · authors · year · venue/status Core idea and why to read it Official resources 04.01 OCR-Free Document Understanding Transformer Donut treats document understanding as direct image-to-sequence generation: a Swin image encoder converts the page into visual tokens and a BART-style autoregressive decoder emits a task-specific structured string. It removes the external OCR engine and its bounding boxes from the inference path, so recognition and semantic parsing are optimized together. “OCR-free” describes deployment; part of pre-training still uses pseudo text produced by a commercial OCR system. Official code 04.02 Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding Pix2Struct pre-trains an image encoder–text decoder by masking regions of web-page screenshots and asking it to generate the corresponding simplified HTML subtree. The page remains pixels: no OCR tokens, DOM tree, or boxes are supplied downstream. A variable-resolution patch sequence preserves aspect ratio under a fixed patch budget, letting one model process documents, charts, UI screenshots, and natural images. — 04.03 ColPali: Efficient Document Retrieval with Vision Language Models ColPali retrieves a document page directly from its rendered image instead of first extracting OCR text, layout blocks, tables, and captions. PaliGemma contextualizes 1,024 visual patch tokens and query text tokens in a shared language-conditioned space; a learned projection keeps one 128-dimensional vector per patch or query token. At search time, ColBERT-style late interaction lets every query token choose its best-matching page patch, then sums those maxima. The page index is computed offline, so fine-grained query–patch matching remains fast enough for retrieval while preserving text, layout, figures, and tables. Official code · Official ViDoRe models and benchmark 04.04 ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering ChartQAPro is a test-only stress benchmark with 1,341 static chart images and 1,948 human-written, human-verified questions. It replaces the narrow source and question distribution of earlier chart QA with 99 represented online sources, dashboards, infographics, multi-chart layouts, optional context paragraphs, five main question categories, and an orthogonal set of unanswerable cases. The headline is not a new model: it is a harder measurement surface for perception, instruction following, and quantitative reasoning. — 04.05 POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering PolyChartQA is a multilingual chart-QA benchmark and companion training corpus that renders chart text, structured data, and questions in ten languages across sixteen chart types. PolyChartQA-Test contains 22,606 chart images and 26,151 QA pairs; PolyChartQA-Train contains 131,515 images and 751,363 QA pairs. Its key scientific question is whether a model that can reason over an English chart still succeeds when the chart itself and the question use other scripts and languages. — Work: Compare OCR-pipeline, end-to-end, and visual-retrieval models under scan, font, language, layout, and schema shifts.
Exit evidence: Report retrieval relevance, exact structure validity, and field-level error under their separate contracts.
2026
Open OCR with structural visual tests
olmOCR combines large-scale supervised document training with reinforcement learning over visual unit tests and an exact structural benchmark; selected tests do not establish complete semantic correctness.
Key papers in this section
Order Primary paper · authors · year · venue/status Core idea and why to read it Official resources 05.01 The olmOCR Project: Building Fully Open OCR using VLMsPeer-reviewed ACL 2026 system demonstration and Best Demonstration Paper award recipient. olmOCR is an open PDF-to-structured-text system built around a 7B vision-language model. PDF pages are rendered as images, paired with document metadata and prompts, decoded into reading-order text with layout and table structure, and validated before assembly. Training combines supervised examples from roughly 260K pages with reinforcement learning over visual unit tests, while olmOCR-Bench checks selected structural properties on about 1.4K documents and 7K tests. Official implementation and models Work: Run olmOCR and OCR-pipeline baselines on pinned document images with exact structural tests for reading order, tables, equations, lists, headers, and serialization, plus blinded semantic error review.
Exit evidence: Report training and test provenance, unit-test coverage, exact and field-level scores, language/layout strata, compute, parser tolerance, contamination checks, and errors not expressible by the tests.
2025–2026
Reasoning budgets, executable outputs and scientific edits
Test-time scaling studies how extra reasoning changes perception and mathematics; EmbedCopilot checks whether plausible embedded-system answers actually execute; GRADE requires domain-correct and readable image edits. Together they motivate measuring compute, implementation success and constraint satisfaction rather than judging prose alone.
Key papers in this section
Order Primary paper · authors · year · venue/status Core idea and why to read it Official resources 06.01 GRADE: Benchmarking Discipline-Informed Reasoning in Image EditingECCV 2026 Spotlight verified in the official Vision-Language Models & Visual Foundation Models session; manuscript #2706 checked on 2026-09-12. Publisher version of record was not independently verified. GRADE evaluates whether an image editor can use disciplinary knowledge to make a scientifically correct visual change, preserve relevant context, and keep the result logically readable. Its 520 image–instruction–reference triplets span ten academic disciplines. The central distinction is between drawing a plausible image and rendering the right answer to a domain-specific problem. ECCV Spotlight program · Conference manuscript · Official project or implementation Work: What external test can falsify a persuasive explanation: a perception answer, a running circuit, or a scientifically valid image?
Exit evidence: More tokens can reduce perceptual accuracy. Judge scores and physical execution use different protocols; GRADE strict accuracy is a conjunction of three maxima, not its mean reasoning score.
2025–2026
Document understanding needs component and layout-aware evaluation
OmniDocBench separates end-to-end parsing from text, formula, table and reading-order evaluation across document types. Matching rules and ignored elements matter as much as the leaderboard. The original paper and later benchmark releases must be cited as distinct versions.
Key papers in this section
Order Primary paper · authors · year · venue/status Core idea and why to read it Official resources 07.01 OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive AnnotationsReview pins the CVPR 2025 paper and supplement (981 pages). The live repository has later data, matching and scoring revisions; its current leaderboard is not the paper benchmark. OmniDocBench evaluates document parsing as structured extraction rather than just readable OCR. It separates text, tables, formulas and reading order, then breaks results down by page type and visual attributes. The paper benchmark contains 981 pages; newer repository versions are separate evaluation configurations. Proceedings PDF · Supplement · Author code and version history Work: Does a better overall parser still fail on the document type or element your application needs?
Exit evidence: The reviewed CVPR paper uses 981 pages; newer releases change coverage and scoring. Parsing fidelity does not establish factual, clinical or scientific correctness of a generated answer.
2026
Retrieve evidence inside the answer-generating model
URaG uses early shared MLLM states for ColBERT-style page ranking, retains selected page tokens and all query tokens, then continues deeper generation. Its staged retrieval and LoRA training connect evidence selection to answer modeling without a separate retrieval backbone.
Key papers in this section
Order Primary paper · authors · year · venue/status Core idea and why to read it Official resources 08.01 URaG: Unified Retrieval and Generation in Multimodal LLMs for Efficient Long Document UnderstandingAAAI 2026 proceedings paper; FLOP reductions are not measured latency. Code repository states non-commercial research use. URaG makes page retrieval an internal operation of a multimodal language model. Early layers contextualize the full document and question; a lightweight head ranks pages, and deeper layers see only selected page tokens plus the query. This preserves fine-grained tokens on retained pages while removing irrelevant pages, rather than compressing every page uniformly or running an independent external retriever. Proceedings PDF · Author code and usage terms Work: Where should evidence be selected so detailed tokens survive without paying full deep-layer cost?
Exit evidence: Joint training always keeps ground-truth evidence pages; inference must select them. FLOP tests extend documents by duplication and are not latency measurements. Multi-page reasoning and WSI transfer need separate evidence.
Research branches
Text recognition and grounding
Which visual token supports each recognized or copied string?
OCR-VQA: Visual Question Answering by Reading Text in Images · Towards VQA Models That Can Read · Iterative Answer Prediction With Pointer-Augmented Multimodal Transformers for TextVQA
Layout and document structure
How are reading order, two-dimensional positions, fields, tables, and hierarchy represented?
LayoutLM: Pre-training of Text and Layout for Document Image Understanding · LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding · DocFormer: End-to-End Transformer for Document Understanding · LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking · PubTables-1M: Towards Comprehensive Table Extraction From Unstructured Documents · DocVQA: A Dataset for VQA on Document Images · OCR-Free Document Understanding Transformer · Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding
Visual document retrieval
Can page-level visual tokens preserve text, layout, tables, and graphics for late-interaction ranking?
ColPali: Efficient Document Retrieval with Vision Language Models
Charts and visualized data
Can the system recover marks, axes, legends, values, and relations?
ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning · ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering · POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering
Open OCR with structural visual tests
Which document properties are enforced by executable tests, and which semantic errors remain outside their coverage?
Reasoning budgets, executable outputs and scientific edits
What external test can falsify a persuasive explanation: a perception answer, a running circuit, or a scientifically valid image?
GRADE: Benchmarking Discipline-Informed Reasoning in Image Editing
Document understanding needs component and layout-aware evaluation
Does a better overall parser still fail on the document type or element your application needs?
OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations
Retrieve evidence inside the answer-generating model
Where should evidence be selected so detailed tokens survive without paying full deep-layer cost?
URaG: Unified Retrieval and Generation in Multimodal LLMs for Efficient Long Document Understanding
Comparison table
| Family or stage | Scientific distinction | Main risk | Anchors |
|---|---|---|---|
| OCR and layout-aware pipeline | Recognizes text first, then combines tokens and two-dimensional boxes, optionally with page pixels and shared multimodal attention. | Recognition and box errors cascade; OCR can still discard non-text evidence. | 2020 · 2020 · 2021 · 2021 · 2022 · 2021 |
| Table detection and structure | Predicts table regions, rows, columns, headers, and spanning cells under a canonical topology. | Dataset conversion and canonicalization can dominate apparent structural accuracy. | 2022 |
| OCR-free generation | Maps pixels directly to serialized target structures. | Decoder validity and memorized templates hide recognition errors. | 2022 · 2023 |
| OCR-free visual retrieval | Embeds each page as multiple visual-language vectors and scores query–page token interactions. | Storage, corpus construction, and relevance labels can dominate retrieval comparisons. | 2025 |
| Chart reasoning | Combines graphical perception, data extraction, and operations. | Template and synthetic-chart shortcuts inflate reasoning scores. | 2022 · 2025 |
| Open OCR with structural visual tests | olmOCR combines large-scale supervised document parsing with reinforcement learning over visual unit tests and an exact structural benchmark. | Optimizing a selected test suite can miss semantically wrong text, unsupported layouts, language failures, or tolerance-sensitive serialization errors. | 2026 |
| Reasoning budgets, executable outputs and scientific edits | Test-time scaling studies how extra reasoning changes perception and mathematics; EmbedCopilot checks whether plausible embedded-system answers actually execute; GRADE requires domain-correct and readable image edits. Together they motivate measuring compute, implementation success and constraint satisfaction rather than judging prose alone. | More tokens can reduce perceptual accuracy. Judge scores and physical execution use different protocols; GRADE strict accuracy is a conjunction of three maxima, not its mean reasoning score. | 2026 |
| Document understanding needs component and layout-aware evaluation | OmniDocBench separates end-to-end parsing from text, formula, table and reading-order evaluation across document types. Matching rules and ignored elements matter as much as the leaderboard. The original paper and later benchmark releases must be cited as distinct versions. | The reviewed CVPR paper uses 981 pages; newer releases change coverage and scoring. Parsing fidelity does not establish factual, clinical or scientific correctness of a generated answer. | 2025 |
| Retrieve evidence inside the answer-generating model | URaG uses early shared MLLM states for ColBERT-style page ranking, retains selected page tokens and all query tokens, then continues deeper generation. Its staged retrieval and LoRA training connect evidence selection to answer modeling without a separate retrieval backbone. | Joint training always keeps ground-truth evidence pages; inference must select them. FLOP tests extend documents by duplication and are not latency measurements. Multi-page reasoning and WSI transfer need separate evidence. | 2026 |
Concept lineage
LayoutLM: Pre-training of Text and Layout for Document Image UnderstandingLayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding
Text plus two-dimensional layout is extended with page-image features and cross-modal pretraining objectives.
LayoutLM: Pre-training of Text and Layout for Document Image UnderstandingDocFormer: End-to-End Transformer for Document Understanding
Layout-aware token modelling is complemented by shared spatial embeddings and multimodal self-attention over text and visual features.
LayoutLMv2: Multi-modal Pre-training for Visually-rich Document UnderstandingLayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking
A multimodal document encoder moves from separate text/image objectives toward unified masking plus explicit word–patch alignment.
PubTables-1M: Towards Comprehensive Table Extraction From Unstructured DocumentsPix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding
Explicit table-topology supervision complements screenshot-to-sequence pretraining, exposing the trade-off between structured labels and general visual parsing.
Towards VQA Models That Can ReadDocVQA: A Dataset for VQA on Document Images
Reading text in images expands into layout- and document-aware question answering.
LayoutLMv2: Multi-modal Pre-training for Visually-rich Document UnderstandingOCR-Free Document Understanding Transformer
An OCR-dependent text–layout–pixel encoder is contrasted with direct OCR-free visual-to-sequence decoding.
OCR-Free Document Understanding TransformerPix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding
OCR-free document decoding broadens into screenshot parsing pretraining over varied visual-language tasks.
DocVQA: A Dataset for VQA on Document ImagesColPali: Efficient Document Retrieval with Vision Language Models
Layout-aware document understanding is complemented by retrieval over page-image tokens using a vision–language model and late interaction.
Failure modes to audit
- Leaking document templates or organizations across splits.
- Scoring normalized strings while ignoring wrong grounding.
- Using OCR text that includes test annotations.
- Treating valid JSON as semantically correct extraction.
- Ignoring multilingual fonts, handwriting, and scan degradation.
Questions for every paper
- Which evidence comes from pixels, OCR, or layout?
- What output schema is required?
- How is grounding verified?
- Which document shifts are tested?
Complete maintained paper route
Section titled “Complete maintained paper route”19 canonical readings · 2019–2026· route reviewed through October 2, 2026.
This is the complete structured list for this chapter, ordered chronologically from earlier foundations toward current work. The explanatory ladder above highlights the shortest conceptual route; this list also includes important branches, benchmarks, and current endpoints. It is generated from the same deduplicated catalog used by the AI background registry and My Library, so a future catalog update appears here automatically.
Show all 19 readings in maintained order
Towards VQA Models That Can Read
Amanpreet Singh et al.
Metadata checked
OCR-VQA: Visual Question Answering by Reading Text in Images
Anand Mishra et al.
Metadata checked
LayoutLM: Pre-training of Text and Layout for Document Image Understanding
Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and Ming Zhou
Official artifacts: Official implementation and models
Metadata checked
Iterative Answer Prediction With Pointer-Augmented Multimodal Transformers for TextVQA
Ronghang Hu et al.
Metadata checked
LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding
Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, and Lidong Zhou
Official artifacts: Official implementation
Metadata checked
DocFormer: End-to-End Transformer for Document Understanding
Srikar Appalaraju, Bhavan Jasani, Bhargava Urala Kota, Yusheng Xie, and R. Manmatha
Metadata checked
DocVQA: A Dataset for VQA on Document Images
Minesh Mathew et al.
Metadata checked
OCR-Free Document Understanding Transformer
Geewook Kim et al.
Official artifacts: Official code
Metadata checked
LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking
Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei
Official artifacts: Official implementation and models
Metadata checked
PubTables-1M: Towards Comprehensive Table Extraction From Unstructured Documents
Brandon Smock, Rohith Pesala, and Robin Abraham
Official artifacts: Official dataset, code, and Table Transformer models
Metadata checked
ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning
Ahmed Masry et al.
Metadata checked
Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding
Kenton Lee et al.
Metadata checked
ColPali: Efficient Document Retrieval with Vision Language Models
Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, Céline Hudelot, and Pierre Colombo
Official artifacts: Official code · Official ViDoRe models and benchmark
Metadata checked
OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations
Linke Ouyang et al.
Review pins the CVPR 2025 paper and supplement (981 pages). The live repository has later data, matching and scoring revisions; its current leaderboard is not the paper benchmark.
Official artifacts: Proceedings PDF · Supplement · Author code and version history
Metadata checked
ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering
Ahmed Masry et al.
Metadata checked
The olmOCR Project: Building Fully Open OCR using VLMs
Jake Poznanski, Kyle Lo, and Luca Soldaini
Peer-reviewed ACL 2026 system demonstration and Best Demonstration Paper award recipient.
Official artifacts: Official implementation and models
Metadata checked
GRADE: Benchmarking Discipline-Informed Reasoning in Image Editing
Mingxin Liu, Ziqian Fan, Zhaokai Wang, Leyao Gu, Zirun Zhu, Yiguo He, Yuchen Yang, Changyao Tian, Xiangyu Zhao, Ning Liao, Shaofeng Zhang, Qibing Ren, Zhihang Zhong, Xuanhe Zhou, Junchi Yan, and Xue Yang
ECCV 2026 Spotlight verified in the official Vision-Language Models & Visual Foundation Models session; manuscript #2706 checked on 2026-09-12. Publisher version of record was not independently verified.
Official artifacts: ECCV Spotlight program · Conference manuscript · Official project or implementation
Metadata checked
URaG: Unified Retrieval and Generation in Multimodal LLMs for Efficient Long Document Understanding
Yongxin Shi, Jiapeng Wang, Zeyu Shan, Dezhi Peng, Zening Lin, and Lianwen Jin
AAAI 2026 proceedings paper; FLOP reductions are not measured latency. Code repository states non-commercial research use.
Official artifacts: Proceedings PDF · Author code and usage terms
Metadata checked
POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering
Yichen Xu et al.
Metadata checked