Writing study: Measuring Agents in Production

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Melissa Pan, Negar Arabzadeh, Riccardo Cogo, Yuxuan Zhu, Alexander Xiong, Lakshya A Agrawal, Huanzhi Mao, Emma Shen, Sid Pallerla, Liana Patel, Shu Liu, Tianneng Shi, Xiaoyuan Liu, Jared Quincy Davis, Emmanuele Lacavalla, Alessandro Basile, Shuyi Yang, Paul Castro, Daniel Kang, Koushik Sen, Dawn Song, Joseph E. Gonzalez, Ion Stoica, Matei Zaharia, Marquita Ellis · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Empirical analysis, diagnosis & auditing (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

47 physical pages

Before references / appendix

≈ 9.04 page-equivalents

Main figures / tables

8 / 0

Attached appendix figures / tables

22 / 5

Attached appendix span

≈ 32.91 page-equivalents

Before references / appendix: ≈ 19.23%References / intervening material: ≈ 10.74%Attached appendix: ≈ 70.02%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Source-bound correction: Source-checked physical-page text restores wrapped top-level heading labels on pp. 4, 5, 7, 8. Heading starts and measured positions are unchanged; this is label completeness, not a new section. Newly detected unnumbered impact/singular-acknowledgment statements are retained from the size-aware source extraction; authored heading corrections are unchanged.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 1.44 pp.	3.06%
2. Related Work	2	≈ 0.33 pp.	0.7%
3. Study Methodology	3	≈ 1.14 pp.	2.43%
4. RQ1 Results: What Are the Applications of Agents?	4	≈ 1.06 pp.	2.26%
5. RQ2 Results: What Models, Architectures, And Techniques Are Used?	5	≈ 1.98 pp.	4.21%
6. RQ3 Results: How Are Agents Evaluated For Deployment?	7	≈ 0.86 pp.	1.83%
7. RQ4 Results: What Are The Top Challenges In Deployment?	8	≈ 1.04 pp.	2.21%
8. Discussions & Conclusion	9	≈ 0.59 pp.	1.26%
Acknowledgements	9	≈ 0.08 pp.	0.17%
Impact Statement	9	≈ 0.15 pp.	0.32%
References	10	≈ 5.04 pp.	10.72%
A. Organization of Appendix	15	≈ 0.23 pp.	0.49%
B. Supplementary Results and Analysis	15	≈ 6.49 pp.	13.81%
C. Collected Data Details	21	≈ 3.97 pp.	8.45%
D. Results on All Survey Data	25	≈ 4.8 pp.	10.21%
E. Terminology	30	≈ 0.11 pp.	0.23%
F. Literature Review Scope and Categorization	30	≈ 0.46 pp.	0.98%
References	31	≈ 0.37 pp.	0.79%
G. Survey Questionnaire Details	31	≈ 16.5 pp.	35.11%

All detected figure and table locations

Main text

• Figure 1 · p. 1
• Figure 2 · p. 2
• Figure 3 · p. 4
• Figure 4 · p. 5
• Figure 5 · p. 6
• Figure 6 · p. 6
• Figure 7 · p. 7
• Figure 8 · p. 8

Attached appendix

• Table 1 · p. 16
• Figure 9 · p. 17
• Figure 10 · p. 18
• Figure 11 · p. 19
• Table 2 · p. 19
• Figure 12 · p. 20
• Figure 13 · p. 21
• Figure 14 · p. 22
• Table 3 · p. 22
• Figure 15 · p. 23
• Figure 16 · p. 23
• Figure 17 · p. 23
• Figure 18 · p. 25
• Figure 19 · p. 26
• Figure 1 · p. 26
• Figure 20 · p. 27
• Figure 21 · p. 27
• Figure 22 · p. 28
• Figure 23 · p. 29
• Figure 24 · p. 29
• Figure 25 · p. 30
• Figure 26 · p. 30
• Table 4 · p. 31
• Figure 27 · p. 32
• Table 5 · p. 32
• Figure 28 · p. 46
• Figure 29 · p. 47

Argument and organization

1. Problem and contribution

   Contrasts primary practitioner evidence with literature surveys and single-system reports, distinguishing 306 valid survey responses, an 86-system production/pilot subset and 20 interviews.

2. Argument sequence

   Four research questions organize results after collection, filtering, human coding and limitations; case identifiers connect numerical findings to deployment explanations.

3. Evidence and verification

   Optional branched questions explain differing response counts; three independent human coders and peer debriefing support interview themes, while stated bootstrap intervals characterize sampling variation rather than remove participation bias.

Problem, claim and contribution

Contrasts primary practitioner evidence with literature surveys and single-system reports, distinguishing 306 valid survey responses, an 86-system production/pilot subset and 20 interviews.

Introduction and methodology, pp. 2-3 · Main paper

Argument and section order

Four research questions organize results after collection, filtering, human coding and limitations; case identifiers connect numerical findings to deployment explanations.

Sections 3-8, pp. 3-9 · Main paper

Figures, captions and visual explanation

Charts print percentages, counts and question-specific N, with captions identifying multi-select questions. Boxed findings help navigation but need the neighboring denominator rather than the headline survey total.

Figure 4, p. 5; Figure 8, p. 8; both PNGs inspected · Main paper

Tables, comparisons and evidence

Optional branched questions explain differing response counts; three independent human coders and peer debriefing support interview themes, while stated bootstrap intervals characterize sampling variation rather than remove participation bias.

Sections 3.1-3.3, pp. 3-4; Appendix G, pp. 31-47 · Main paper

Appendix and reproducibility

The appendix preserves unfiltered-stage results alongside deployed-only analyses and exposes survey flow and wording, allowing readers to inspect selection and missingness rather than assume one common denominator.

Appendix D, pp. 25-29; G, pp. 31-47 · Main paper

Clarity and writing conventions

Results distinguish survey reports from interview interpretations, but a local prose/chart mismatch remains: p. 8 calls rule-based use 42%, while Figure 8 reports 38.7% (12/31); cross-referencing is 41.9%.

Section 6.2 and Figure 8, p. 8 · Main paper

What to reuse—and what not to infer

Transferable lesson: In a mixed-methods paper, carry the evidence source, item-specific denominator and sampling limits into every compressed finding.

Boundary: Participation and network sampling, optional answers and the April-November 2025 window limit generalization; reported practices do not prove that a design causes production reliability or represent all deployed agents.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-focused distributed manuscript scan, not exhaustive technical/proof validation. Sections 1-8, pp. 1-9: research questions, interview/survey methods and limitations, applications, engineering choices, evaluation practices, reliability themes and discussion.

Appendix / supplement

Writing-focused appendix scan; code, data and external assets not executed. Appendices A-G, pp. 15-47: organization, detailed findings, case/survey provenance, unfiltered results, terminology, related work, question text and survey branching diagrams.

Visually inspected PDF pages

5, 8

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published ICML 2026 proceedings PDF, PMLR volume 306; retrieved 2026-10-07. Public OpenReview decision notes were not independently reverified. · 47 pages