Writing study: Pretrained Vision-Language-Action Models are Surprisingly Resistant to Forgetting in Continual Learning

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Huihan Liu, Changyeon Kim, Bo Liu, Minghuan Liu, Yuke Zhu · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Empirical analysis, diagnosis & auditing (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

22 physical pages

Before references / appendix

≈ 9.69 page-equivalents

Main figures / tables

8 / 5

Attached appendix figures / tables

6 / 7

Attached appendix span

≈ 8.92 page-equivalents

Before references / appendix: ≈ 44.05%References / intervening material: ≈ 15.41%Attached appendix: ≈ 40.55%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Source-bound correction: Include the actual p14 appendix contents cover and complete wrapped main headings. Remove a p15 in-prose reference to main Figure1 that was counted as a new appendix figure.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	2	≈ 0.64 pp.	2.91%
2. Related Work	2	≈ 0.57 pp.	2.59%
3. Preliminaries	3	≈ 0.55 pp.	2.5%
4. VLAs are Surprisingly Resistant to Forgetting	3	≈ 1.94 pp.	8.82%
5. Pretraining Plays an Integral Role in Improving Continual Learning Performance	5	≈ 1.96 pp.	8.91%
6. VLAs Retain Knowledge that is Seemingly Forgotten	7	≈ 1.67 pp.	7.59%
7. Conclusion and Discussion	9	≈ 0.65 pp.	2.95%
Acknowledgment	10	≈ 0.17 pp.	0.77%
Impact Statement	10	≈ 0.47 pp.	2.14%
References	10	≈ 3.4 pp.	15.45%
Appendix: Table of Contents	14	≈ 1 pp.	4.55%
A. More Continual Learning Results	15	≈ 2.64 pp.	12%
B. Experimental Setup Details	17	≈ 2.41 pp.	10.95%
C. Study on Other Factors that Contribute to VLA’s Continual Learning Behavior	20	≈ 0.47 pp.	2.14%
D. Per-Task Knowledge Transfer Curves: Pi0 vs. BC-Transformer	20	≈ 2.39 pp.	10.86%

All detected figure and table locations

Main text

• Figure 1 · p. 1
• Table 1 · p. 4
• Figure 2 · p. 4
• Table 2 · p. 5
• Table 3 · p. 5
• Figure 3 · p. 6
• Figure 4 · p. 6
• Table 4 · p. 7
• Figure 5 · p. 7
• Figure 6 · p. 8
• Figure 7 · p. 8
• Table 5 · p. 8
• Figure 8 · p. 9

Attached appendix

• Figure 9 · p. 15
• Figure 10 · p. 16
• Table 6 · p. 17
• Table 7 · p. 17
• Table 8 · p. 17
• Table 9 · p. 19
• Table 10 · p. 19
• Table 11 · p. 20
• Table 12 · p. 20
• Figure 11 · p. 21
• Figure 12 · p. 21
• Figure 13 · p. 22
• Figure 14 · p. 22

Argument and organization

1. 1 Introduction; 2 Related Work; 3 Preliminaries

   Introduces continual-policy learning and separates success from forgetting metrics.

2. 4 Resistance to Forgetting; 5 Role of Pretraining

   Tests replay size, algorithm choice and matched-architecture pretraining variants.

3. 6 Knowledge Retention; 7 Conclusion

   Uses component swaps and relearning to investigate apparently lost capabilities before scope limits.

Problem, claim and contribution

The broad resistance-to-forgetting title is made conditional in the abstract and experiments on experience replay; sequential training and EWC do not inherit the same finding.

Abstract, p.1; Table 2, p.5 · Main paper

Argument and section order

The analysis separates low forgetting from successful new-task learning before turning to the more mechanistic swap and recovery probes.

Sections 5-6, pp.5-8 · Main paper

Figures, captions and visual explanation

The opening heatmap grid makes task/checkpoint evolution visible across four suites; the later three-part diagram explicitly contrasts normal training, component swaps and relearning.

Figure 1, p.1; Figure 6, p.8 · Main paper

Tables, comparisons and evidence

The recovery result counts optimization steps while reusing the full original-task dataset, and the real-robot pilot is explicitly only three tasks in a single run.

Real-World Evaluation, p.5; recovery protocol and limitations, p.9 · Main paper

Appendix and reproducibility

Normalized forgetting exposes how a lower initial success rate can misleadingly improve absolute NBT; additional model-size and loss controls are labeled initial investigations.

Appendix A.2, p.16; Appendix C, p.20 · Main paper

Clarity and writing conventions

Knowledge retention is operationalized through faster relearning, while the discussion acknowledges benchmark concentration and replay retention costs.

Section 6, pp.8-9 · Main paper

What to reuse—and what not to infer

Transferable lesson: Pair retention metrics with acquisition quality, and distinguish rapid relearning in steps from learning with fewer examples.

Boundary: Replay remains important; component swaps and recovery are behavioral probes rather than direct proof of preserved representations, and most evidence shares LIBERO’s setting.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-oriented scan of the main argument through physical p.9, with focused physical-page text inspection on 1, 3, 4, 5, 7, 8, 9. Framing, section progression, mechanism/proof explanation, displays, comparison conditions and conclusions were inspected. This is not exhaustive technical reading, proof verification or reproduction.

Appendix / supplement

Appendix organization and main-text pointers were checked; focused text inspection of physical pages 16, 20. Remaining appendix proofs and details were not exhaustively read. No separately linked supplement inspected.

Visually inspected PDF pages

1, 8

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published proceedings PDF, PMLR volume 306, retrieved 2026-10-07 · 22 pages