Writing study: VenusBench-Mobile: A Challenging and User-Centric Benchmark for Mobile GUI Agents with Capability Diagnostics

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Yichen Gong, Zhuohan Cai, Sunhao Dai, Yuqi Zhou, Zhangxuan Gu, Changhua Meng, Shuheng Shen · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Datasets, benchmarks & measurement (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

26 physical pages

Before references / appendix

≈ 8.72 page-equivalents

Main figures / tables

6 / 5

Attached appendix figures / tables

4 / 3

Attached appendix span

≈ 15.91 page-equivalents

Before references / appendix: ≈ 33.54%References / intervening material: ≈ 5.27%Attached appendix: ≈ 61.19%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 1.09 pp.	4.19%
2. Related Work	2	≈ 0.53 pp.	2.04%
3. VenusBench-Mobile Benchmark	3	≈ 3 pp.	11.54%
4. Evaluation Results and Analysis	6	≈ 3.03 pp.	11.65%
5. Conclusion	9	≈ 0.23 pp.	0.88%
Acknowledgements	9	≈ 0.06 pp.	0.23%
Impact Statement	9	≈ 0.05 pp.	0.19%
References	9	≈ 1.37 pp.	5.27%
A. Four Proficiency Levels Across Five Dimensions	11	≈ 3.13 pp.	12.04%
B. Benchmark Setting Details	14	≈ 7.64 pp.	29.38%
C. Detailed Information of Fine-grained Capabilities	21	≈ 0.53 pp.	2.04%
D. Agent Framework	22	≈ 2.15 pp.	8.27%
E. Future Work	24	≈ 2.47 pp.	9.5%

All detected figure and table locations

Main text

• Figure 1 · p. 1
• Table 1 · p. 3
• Figure 2 · p. 4
• Figure 3 · p. 4
• Figure 4 · p. 4
• Table 2 · p. 5
• Figure 5 · p. 6
• Table 3 · p. 7
• Figure 6 · p. 7
• Table 4 · p. 8
• Table 5 · p. 9

Attached appendix

• Figure 7 · p. 18
• Figure 8 · p. 19
• Figure 9 · p. 20
• Figure 10 · p. 21
• Table 6 · p. 25
• Table 7 · p. 26
• Table 8 · p. 26

Argument and organization

1. Problem and contribution

   Contrasts app-function-centric benchmarks with user-intent-driven tasks and motivates diagnostic categories beyond one aggregate success score.

2. Argument sequence

   Task categories and capability levels precede evaluation rules; results reuse these categories and then isolate perturbation stability as a separate test.

3. Evidence and verification

   The stability subset has 20 tasks and reports minimum, maximum and joint pass rate, preventing an average condition score from standing in for consistent task completion.

Problem, claim and contribution

Contrasts app-function-centric benchmarks with user-intent-driven tasks and motivates diagnostic categories beyond one aggregate success score.

Introduction, pp. 1-2 · Main paper

Argument and section order

Task categories and capability levels precede evaluation rules; results reuse these categories and then isolate perturbation stability as a separate test.

Sections 3-4, pp. 3-9 · Main paper

Figures, captions and visual explanation

The taxonomy page pairs named categories with interface examples, then uses distribution and capability-level heatmaps. The stability table distinguishes individual settings from success across all five.

Figures 2-4, p. 4; Table 4, p. 8; both PNGs inspected · Main paper

Tables, comparisons and evidence

The stability subset has 20 tasks and reports minimum, maximum and joint pass rate, preventing an average condition score from standing in for consistent task completion.

Figure 3, p. 4; Table 4 and Section 4.4, p. 8 · Main paper

Appendix and reproducibility

Judge templates are differentiated by operation and include positive/negative visual examples; an action-space table specifies permissible interactions separately from capability results.

Appendix B.4, pp. 14-21; Tables 7-8, p. 26 · Main paper

Clarity and writing conventions

The results use explicit finding summaries, but the claim that harder-level performance proves taxonomy validity exceeds what a performance trend alone establishes.

Section 4.3, p. 8; Table 8, p. 26 · Main paper

What to reuse—and what not to infer

Transferable lesson: Keep per-condition performance and across-condition reliability separate, with the success predicate and subset size visible.

Boundary: Capability labels and score trends are not independent causal diagnoses. Table 8 includes advanced-level exceptions, so avoid describing every dimension as monotonically harder.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-focused distributed manuscript scan, not exhaustive technical/proof validation. Sections 1-5, pp. 1-9: user-intent motivation, task/capability taxonomies, evaluation infrastructure, diagnostic results, stability and inference cost.

Appendix / supplement

Writing-focused appendix scan; code, data and external assets not executed. Appendices A-E, pp. 11-26: proficiency definitions, curation/action space, task-specific judge prompts and examples, detailed capability results, agent framework and future work.

Visually inspected PDF pages

4, 8

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published ICML 2026 proceedings PDF, PMLR volume 306; retrieved 2026-10-07. Public OpenReview decision notes were not independently reverified. · 26 pages