Writing study: Computer Use at the Edge of the Statistical Precipice

NeurIPS overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← NeurIPS 2026 oral writing corpus

Pierluca D'Oro, Sneha Silwal, William R Wong, Yuxuan Sun, Fanyi Xiao, Manchen Wang, Eric Gan, Allen Bolourchi, Joseph Tighe · Official NeurIPS 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Empirical analysis, diagnosis & auditing (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

24 physical pages

Before references / appendix

≈ 9.39 page-equivalents

Main figures / tables

3 / 2

Attached appendix figures / tables

5 / 5

Attached appendix span

≈ 12.93 page-equivalents

Before references / appendix: ≈ 39.13%References / intervening material: ≈ 7%Attached appendix: ≈ 53.87%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Source-bound correction: Top-level heading and reference/appendix boundaries checked against source PDF text. Bibliography titles/author initials, numbered contribution/property bullets, diagram steps, equations and TOC entries are not substantive headings. Section spans are page-coordinate estimates, not ink area or prose-only length. This is an author manuscript mirror, not a verified accepted-version template. Source layout uses physical single-column y/page-height, including equation-heavy pages. Complete numbered-caption inventory cross-checked against source text; restored real punctuation-free captions and excluded inline references. Selected pages visually inspected, not every image/table.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 Introduction	1	≈ 1.04 pp.	4.33%
2 Common pitfalls of CUA environment and metric design	2	≈ 1.81 pp.	7.54%
3 Principles for Computer Use Environment Design	4	≈ 1.08 pp.	4.5%
4 From point estimates to principled aggregation	5	≈ 1.15 pp.	4.79%
5 DigiWorld: Following PRISM for Mobile Computer Use	6	≈ 1.14 pp.	4.75%
6 Experiments	7	≈ 1.79 pp.	7.46%
7 Related work	9	≈ 0.51 pp.	2.13%
8 Limitations, broader impact, and conclusion	10	≈ 0.31 pp.	1.29%
References	10	≈ 1.68 pp.	7%
Appendix	12	≈ 0.05 pp.	0.21%
A Proof of Replay Equivalence	12	≈ 0.28 pp.	1.17%
B Replay agent evaluation details	12	≈ 0.3 pp.	1.25%
C Cost considerations	12	≈ 0.39 pp.	1.63%
D Additional simulation results	13	≈ 0.38 pp.	1.58%
E Detailed DigiWorld statistics	13	≈ 0.25 pp.	1.04%
F Sources of variability in DigiWorld	13	≈ 0.04 pp.	0.17%
G Scenario lifecycle details	13	≈ 1.02 pp.	4.25%
H Variability decomposition: methodology and per-app details	14	≈ 0.96 pp.	4%
I Per-app performance	15	≈ 0.04 pp.	0.17%
J Additional model results	15	≈ 0.1 pp.	0.42%
K Integrity checking implementation details	15	≈ 1.9 pp.	7.92%
L Detailed PRISM evaluation of existing benchmarks	17	≈ 3.84 pp.	16%
M Simulation calibration from evaluation data	21	≈ 0.93 pp.	3.88%
N Application gallery	22	≈ 0.13 pp.	0.54%
O Visual impact of variability axes	22	≈ 2.34 pp.	9.75%

All detected figure and table locations

Main text

• Table 1 · p. 3
• Table 2 · p. 5
• Figure 1 · p. 7
• Figure 2 · p. 8
• Figure 3 · p. 9

Attached appendix

• Table 3 · p. 13
• Table 4 · p. 13
• Table 5 · p. 14
• Table 6 · p. 14
• Figure 4 · p. 15
• Table 7 · p. 15
• Figure 5 · p. 16
• Figure 6 · p. 17
• Figure 7 · p. 23
• Figure 8 · p. 24

Argument and organization

1. Problem and contribution

   Turns benchmark reliability into a falsifiable concern through an intentionally weak replay agent, then asks which sources of variability the reported uncertainty actually includes.

2. Argument sequence

   Separates design principles, variability sources, statistical estimators, a concrete benchmark implementation and validation experiments before related work and limitations.

3. Evidence and verification

   The caption states the simulation calibration and defines expected deployment regret; the text distinguishes per-configuration coverage from suite-level coverage and links each to a separate estimator.

Problem, claim and contribution

Turns benchmark reliability into a falsifiable concern through an intentionally weak replay agent, then asks which sources of variability the reported uncertainty actually includes.

Sections 1–2, pp. 1–3 · Main paper

Argument and section order

Separates design principles, variability sources, statistical estimators, a concrete benchmark implementation and validation experiments before related work and limitations.

Sections 3–8, pp. 4–10 · Main paper

Figures, captions and visual explanation

Figure 1 combines a nominal-coverage reference line, a nested-resampling bar comparison and a deployment-regret waterfall. Each panel tests a different link in the evaluation argument, rather than repeating the same headline accuracy.

Figure 1, p. 7 (render inspected) · Main paper

Tables, comparisons and evidence

The caption states the simulation calibration and defines expected deployment regret; the text distinguishes per-configuration coverage from suite-level coverage and links each to a separate estimator.

Sections 4.2–4.3 and 6.1–6.2, pp. 6–8 · Main paper

Appendix and reproducibility

Detailed benchmark assessments, reset/verification mechanics and simulation assumptions let readers inspect why a seemingly large collection of rollouts can still have a small effective sample size.

Appendices D–M, pp. 13–24 · Main paper

Clarity and writing conventions

Named principles and run-in labels such as scenario lifecycle and integrity at scale make the distinction between benchmark construction and statistical reporting explicit. The rendered method page places the Wilson-interval formula beside its small-sample motivation, then uses bold subheads to separate suite aggregation from hierarchical resampling. The text explicitly treats the curated app set as fixed while resampling within apps, a crucial distinction for interpreting the next page's coverage visual.

Sections 3 and 5, pp. 4–7; Sections 4.2–4.3, p. 6 (additional rendered inspection) · Main paper

What to reuse—and what not to infer

Transferable lesson: Align the uncertainty calculation with the hierarchy that generated the data, and show how a reporting choice changes a concrete model-selection decision.

Boundary: The simulated coverage and illustrative regret do not establish reliability for every computer-use benchmark. Keep calibration assumptions and the particular app/model comparison visible.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-02

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-oriented scan across manuscript framing, section sequence, figure/table captions, selected argument/evaluation prose and conclusion; not exhaustive technical or proof verification. Sections 1–8, pp. 1–10; reference boundary pp. 10–11. Source is the exact public author version listed below, not a verified accepted/camera-ready revision.

Appendix / supplement

Writing-oriented appendix/documentation scan: Appendices beginning on p. 12 through p. 24: replay argument, confidence intervals, costs, benchmark assessment, integrity checks and simulation settings. Source code, data, model outputs and external video assets were not executed or independently validated.

Visually inspected PDF pages

6, 7

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Public author arXiv manuscript 2605.08261v1; not verified identical to the accepted or camera-ready OpenReview revision · 24 pages