Writing study: Scaling Laws for Native Multimodal Models

ICCV overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICCV 2025 oral writing corpus

Mustafa Shukor, Enrico Fini, Victor Guilherme Turrisi da Costa, Matthieu Cord, Joshua Susskind, Alaaeldin El-Nouby · Official ICCV 2025 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Empirical analysis, diagnosis & auditing (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

12 physical pages

Before references / appendix

≈ 8.12 page-equivalents

Main figures / tables

12 / 4

Attached appendix figures / tables

0 / 0

Attached appendix span

No boundary detected; not assumed zero

Before references / appendix: ≈ 67.67%References / intervening material: ≈ 32.33%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Extraction notes: No attached appendix boundary detected; Figure numbering discontinuity; see source note. Zero means no recognized caption, not independently verified absence.

Source-bound correction: Source numbering jumps from Figure5 to Figure7 on physical p.5; p.4 contains Figures3–4. Retain the genuine numbering gap: 12 distinct main figure labels, not 13 inferred from the highest number. No synthetic Figure6 is added.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 1.34 pp.	11.17%
2. Preliminaries	2	≈ 1.03 pp.	8.58%
3. Scaling native multimodal models	3	≈ 2.57 pp.	21.42%
4. Towards multimodal specialization	6	≈ 1.45 pp.	12.08%
5. Evaluation on downstream tasks with SFT	7	≈ 0.12 pp.	1%
6. Related work	7	≈ 0.9 pp.	7.5%
7. Limitations	8	≈ 0.1 pp.	0.83%
8. Conclusion	8	≈ 0.19 pp.	1.58%
Acknowledgment	9	≈ 0.07 pp.	0.58%
References	9	≈ 3.88 pp.	32.33%

All detected figure and table locations

Main text

• Figure 1 · p. 1
• Table 1 · p. 2
• Figure 2 · p. 3
• Table 2 · p. 3
• Figure 3 · p. 4
• Figure 4 · p. 4
• Figure 5 · p. 5
• Table 3 · p. 5
• Figure 7 · p. 5
• Figure 8 · p. 6
• Figure 9 · p. 6
• Figure 10 · p. 7
• Table 4 · p. 7
• Figure 11 · p. 7
• Figure 12 · p. 8
• Figure 13 · p. 8

Attached appendix

No recognized caption labels.

Argument and organization

1. 1 Introduction and findings; 2 Preliminaries

   States the early-versus-late fusion question, then defines native training, routing, compute and the scaling-law model.

2. 3 Scaling native multimodal models; 4 Towards multimodal specialization

   Organizes the narrative by controlled changes in architecture, data mixture, initialization and routing.

3. 5 Downstream tasks; 6 Related work; 7 Limitations; 8 Conclusion

   Checks loss-based rankings after instruction tuning before limiting extrapolation and downstream interpretation.

Problem, claim and contribution

Asks whether late fusion is intrinsically preferable when all modalities are trained from scratch, rather than treating pretrained backbones as a neutral baseline.

Abstract and Section 1, physical pp.1–2 · Main paper

Argument and section order

A findings summary precedes definitions and setup; finding-led subsections reuse fixed-compute and active-parameter distinctions.

Sections 1.1–4, pp.2–7 · Main paper

Figures, captions and visual explanation

Paired scaling plots keep modality panels and parameter-size encodings consistent; the fitted surface and coefficient table separate measured points from the fitted relationship.

Figure 2 and Table 2, p.3; Figures 10–11, p.7 · Main paper

Tables, comparisons and evidence

The downstream table states 1.5B scale and 300B pretraining tokens, separates accuracy from CIDEr, and retains tasks where the overall-leading method is not best.

Table 4 and Section 5, p.7 · Main paper

Appendix and reproducibility

Implementation, alternative vision-encoder allocations, scaling derivations and extrapolation discussion are routed to Appendices A–D; only main-paper pointers were inspected.

Sections 2.3, 3.1, 4.2 and 7, pp.3–8 · Main paper

Clarity and writing conventions

The limitations explicitly separate predictable validation-loss scaling from uncertain downstream transfer and extreme-scale extrapolation.

Section 7, p.8 · Main paper

What to reuse—and what not to infer

Transferable lesson: Define the resource contract before presenting scaling curves, then include a downstream check and an explicit extrapolation boundary.

Boundary: FLOPs and active parameters are modeled resource proxies, not interchangeable with wall-clock cost or total MoE storage. The setup includes private HQITP data; the small downstream check does not establish universal architecture superiority.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-oriented scan across the main manuscript, physical PDF pp.1–8, covering framing, section progression, method explanations, captions, comparison protocols and concluding scope; reference boundary checked. Selected full-page rasters were inspected. Not exhaustive line-by-line reading, proof verification or experimental reproduction.

Appendix / supplement

Main-paper appendix/supplement references were checked. A separate supplementary source is linked in the inventory but was not retrieved or inspected; its contents are not included in the reading or visual-page counts.

Visually inspected PDF pages

3, 7

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Official CVF open-access ICCV 2025 accepted-paper PDF, retrieved 7 October 2026; CVF states identical to accepted version except watermark; not assumed identical to initial submission · 12 pages