Writing study: SWINGARENA: Adversarial Programming Arena for Long-context GitHub Issue Solving

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Wendong XU, Jing Xiong, Chenyang Zhao, Qiujiang Chen, Haoran Wang, Hui Shen, Zhongwei Wan, Jianbo Dai, Taiqiang Wu, He Xiao, Chaofan Tao, Zhuoqing Mao, Ying Sheng, Zhijiang Guo, Hongxia Yang, Bei Yu, Lingpeng Kong, Quanquan Gu, Ngai Wong · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Datasets, benchmarks & measurement resources (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

29 physical pages

Before references / appendix

≈ 10.11 page-equivalents

Main figures / tables

3 / 3

Attached appendix figures / tables

4 / 3

Attached appendix span

≈ 15.73 page-equivalents

Before references / appendix: ≈ 34.86%References / intervening material: ≈ 10.9%Attached appendix: ≈ 54.24%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 1.65 pp.	5.69%
2 RELATED WORK	3	≈ 0.59 pp.	2.03%
3 SWINGARENA	3	≈ 3.34 pp.	11.52%
4 EXPERIMENTS	7	≈ 2.96 pp.	10.21%
5 CONCLUSION	10	≈ 0.91 pp.	3.14%
REFERENCES	11	≈ 3 pp.	10.34%
REPRODUCIBILITY STATEMENT	14	≈ 0.16 pp.	0.55%
A MORE EXPERIMENT RESULTS IN SWINGARENA	14	≈ 0.22 pp.	0.76%
B DATA FEATURE DISTRIBUTION	14	≈ 2.03 pp.	7%
C DATA ANALYSIS	16	≈ 2.59 pp.	8.93%
D LLM EVALUATION WORKFLOW IN SWINGARENA	19	≈ 0.3 pp.	1.03%
E ANNOTATOR DEMOGRAPHICS	19	≈ 0.37 pp.	1.28%
F LIMITATION	19	≈ 0.69 pp.	2.38%
G BROADER IMPACT	20	≈ 0.64 pp.	2.21%
H PROMPTS USED IN SWINGARENA	21	≈ 3.72 pp.	12.83%
I STEP RESULTS	24	≈ 5.17 pp.	17.83%

All detected figure and table locations

Main text

• Figure 1 · p. 3
• Figure 2 · p. 5
• Table 1 · p. 8
• Table 2 · p. 8
• Figure 3 · p. 9
• Table 3 · p. 9

Attached appendix

• Figure 4 · p. 14
• Table 4 · p. 14
• Figure 5 · p. 15
• Figure 6 · p. 16
• Figure 7 · p. 16
• Table 5 · p. 17
• Table 6 · p. 19

Argument and organization

1. From static tests to interactive evaluation

   Real development workflow motivates data filtering, submitter–reviewer roles and CI gates.

2. Arena, retrieval and failure analysis

   Protocol and context access are separated before matchup, language and ablation evidence.

Problem, claim and contribution

The introduction identifies three concrete evaluation omissions—static challenges, isolated agents and missing quality gates—then ties contributions to an interactive protocol rather than claiming a new retrieval algorithm.

Section 1, page 2 · Main paper

Figures, captions and visual explanation

The overview combines retrieval, patch verification, test verification and role switching with different colored branches. It conveys workflow completeness, although small embedded code and many arrows make it less readable at reduced width.

Figure 2, page 5 · Main paper

Tables, comparisons and evidence

Matchup rows explicitly identify both agents and separate reviewer, submitter and adversarial-win metrics. This is preferable to ranking models by a single win rate that also depends on opponent strength.

Table 1, page 8; metric definitions, page 7 · Main paper

The displayed Top-20 file-hit rates are below Top-10 rates despite being described as cumulative correct-file retrieval fractions. Ranking regeneration or a different denominator would need explanation before treating this as recall@k.

Tables 5–6, pages 17 and 19 · Main paper

Appendix and reproducibility

Prompts, structured input/output and CI-log examples expose operational choices; annotated failure categories connect retrieval limitations to concrete multi-file and non-functional failures.

Appendices C, H and I, pages 17–18 and 21–29 · Main paper

Clarity and writing conventions

Repetition of the battle protocol and failure taxonomy lengthens the manuscript without adding definitions; several cross-references and open-source matchup labels do not agree with their destinations or row contents.

Sections 3.2–3.3; Appendix C; Table 4, page 14 · Main paper

What to reuse—and what not to infer

Transferable lesson: For interactive benchmarks, diagram every actor and acceptance gate, distinguish opponent-dependent scores from standalone success, and publish complete protocol examples.

Boundary: Some metric aggregation and provenance remain unclear: win rates can exceed average check-pass rates, cumulative hit-rate columns decrease, and prompt/CI examples are not fully consistent with stated schemas. These are clarification needs, not independently replicated failures.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

Pages 1–10, Sections 1–5

Appendix / supplement

Pages 14–29, statements and Appendices A–I including prompts, JSON examples and CI output

Visually inspected PDF pages

5, 8

• ICLR 2026 final proceedings · 29 pages