Writing study: RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Zeyi Liao, Jaylen Jones, Linxi Jiang, Yuting Ning, Eric Fosler-Lussier, Yu Su, ZHIQIANG LIN, Huan Sun · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Datasets, benchmarks & measurement resources (primary) · Empirical analysis, diagnosis & auditing.

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

46 physical pages

Before references / appendix

≈ 10.74 page-equivalents

Main figures / tables

3 / 3

Attached appendix figures / tables

7 / 10

Attached appendix span

≈ 28.89 page-equivalents

Before references / appendix: ≈ 23.35%References / intervening material: ≈ 13.85%Attached appendix: ≈ 62.8%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 1.82 pp.	3.96%
2 BACKGROUND	3	≈ 0.87 pp.	1.89%
3 REDTEAMCUA - HYBRID ENVIRONMENT SANDBOX	4	≈ 1.01 pp.	2.2%
4 ADVERSARIAL TESTING WITH REDTEAMCUA	5	≈ 1.25 pp.	2.72%
5 BENCHMARKING CUAS AGAINST INDIRECT PROMPT INJECTION	6	≈ 2.46 pp.	5.35%
6 ANALYSIS	9	≈ 0.39 pp.	0.85%
7 END2END EVALUATION	9	≈ 1.24 pp.	2.7%
8 CONCLUSION	10	≈ 0.51 pp.	1.11%
ETHICS STATEMENT	11	≈ 0.26 pp.	0.57%
REPRODUCIBILITY STATEMENT	11	≈ 0.15 pp.	0.33%
LLM USAGE STATEMENT	11	≈ 0.08 pp.	0.17%
REFERENCES	11	≈ 6.36 pp.	13.83%
A LIMITATIONS	18	≈ 1 pp.	2.17%
B RTC-BENCH	19	≈ 11 pp.	23.91%
C REDTEAMCUA FRAMEWORK DIAGRAM	30	≈ 1.53 pp.	3.33%
D COMPARISON OF CUA EVALUATION FRAMEWORKS	31	≈ 1.56 pp.	3.39%
E EXPERIMENT SETUP DETAILS	33	≈ 0.18 pp.	0.39%
F CIA CLASSIFICATION PRINCIPLES	33	≈ 0.12 pp.	0.26%
G LICENSES	33	≈ 0.12 pp.	0.26%
H DEFENSE RESULTS	33	≈ 1.87 pp.	4.07%
I ADDITIONAL RESULTS	35	≈ 7.61 pp.	16.54%
J SYSTEM PROMPTS	43	≈ 3.89 pp.	8.46%

All detected figure and table locations

Main text

• Figure 1 · p. 2
• Table 1 · p. 7
• Figure 2 · p. 8
• Figure 3 · p. 9
• Table 3 · p. 10
• Table 2 · p. 10

Attached appendix

• Figure 4 · p. 19
• Figure 5 · p. 20
• Figure 6 · p. 21
• Table 4 · p. 23
• Table 5 · p. 25
• Figure 7 · p. 30
• Table 6 · p. 31
• Table 7 · p. 34
• Figure 8 · p. 34
• Table 8 · p. 36
• Table 9 · p. 37
• Table 10 · p. 37
• Figure 9 · p. 38
• Table 11 · p. 39
• Table 12 · p. 40
• Figure 10 · p. 40
• Table 13 · p. 41

Argument and organization

1. Threat and evaluation gap

   Define why hybrid web/OS use needs a realistic but isolated security benchmark.

2. Sandbox and benchmark construction

   Separate the reusable environment from its task and attack taxonomy.

3. Decoupled evaluation, defenses, and end-to-end testing

   Distinguish susceptibility after exposure from navigation and successful harmful execution.

4. Appendix artifacts

   Provide scenarios, prompts, evaluators, ablations, and outcome categories for reproduction.

Problem, claim and contribution

The paper defines the benign workflow, attacker control, and environmental gap before introducing the benchmark. The 9×24×4 construction makes the 864-example count interpretable rather than using scale alone as novelty.

Sections 2–4, pages 3–6. · Main paper

Argument and section order

Decoupled exposure is introduced as a diagnostic, not a replacement for realistic use. A separate end-to-end section then tests whether navigation failures were masking risk.

Sections 3.1, 5.1, and 7, pages 5–10. · Main paper

Figures, captions and visual explanation

The framework diagram separates benign/adversarial configuration and evaluators while showing their shared environment. Concrete interface screenshots anchor the abstraction, and the caption points to larger versions.

Figure 7, page 30; Appendix B.1, pages 19–22. · Main paper

Tables, comparisons and evidence

Attempt rate and execution-verified success rate distinguish willingness from capability. The manually coded outcome table further separates detection, navigation failure, incomplete execution, and successful attacks, explaining aggregate differences.

Table 1, page 7; Table 3, page 10; Appendix I.6, page 42. · Main paper

Appendix and reproducibility

The contents page supports a long, artifact-heavy supplement; full prompts, scenario examples, action translation, step budgets, and evaluator details let readers understand the experimental scaffold rather than only model names.

Page 17; Appendices B, E, and J, pages 19–29, 33, and 43–46. · Main paper

Clarity and writing conventions

Captions retain aggregation rules and metric direction. The strongest headline, however, needs the subset and denominator stated nearby: end-to-end testing is on selected high-risk tasks, not the full 864 examples.

Table 1 caption, page 7; Section 7 and Table 3, pages 9–10. · Main paper

What to reuse—and what not to infer

Transferable lesson: For an agent benchmark, explicitly separate exposure, attempted behavior, successful execution, and environmental navigation; accompany headline rates with denominators, selection rules, and concrete failure categories.

Boundary: The 50 end-to-end/defense tasks are selected for high attack success, so their rates are not population risk estimates or directly comparable to full-benchmark averages. Opus 4.5 uses 38/46, not 38/50. Operator results depend on simulated human-check handling; attempt rate uses an LLM judge. Source prompts and attack examples are research artifacts, not instructions to execute.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

Pages 1–11, Sections 1–8 and ethics/reproducibility statements; references excluded from analytical reading.

Appendix / supplement

Pages 17–46, contents and Appendices A–J, including benchmark examples, comparison framework, defense and ablation results, and full agent/evaluator prompts.

Visually inspected PDF pages

10, 30

• Official ICLR 2026 proceedings conference PDF; PDF creation metadata: Sun Mar 1 02:09:53 2026 EST · 46 pages