Writing study: Verifying Chain-of-Thought Reasoning via Its Computational Graph

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Zheng Zhao, Yeskendir Koishekenov, Xianjun Yang, Naila Murray, Nicola Cancedda · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Empirical analysis, diagnosis & auditing (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

28 physical pages

Before references / appendix

≈ 9.88 page-equivalents

Main figures / tables

4 / 4

Attached appendix figures / tables

4 / 15

Attached appendix span

≈ 12.89 page-equivalents

Before references / appendix: ≈ 35.29%References / intervening material: ≈ 18.68%Attached appendix: ≈ 46.04%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 0.95 pp.	3.39%
2 PROBLEM FORMULATION AND PRELIMINARIES	2	≈ 1.09 pp.	3.89%
3 METHODOLOGY	3	≈ 1.7 pp.	6.07%
4 EXPERIMENTS	5	≈ 4.09 pp.	14.61%
5 RELATED WORK	9	≈ 0.87 pp.	3.11%
6 CONCLUSION	10	≈ 0.18 pp.	0.64%
ETHICS STATEMENT	10	≈ 0.18 pp.	0.64%
REPRODUCIBILITY STATEMENT	10	≈ 0.13 pp.	0.46%
ACKNOWLEDGMENTS	10	≈ 0.11 pp.	0.39%
REFERENCES	10	≈ 5.23 pp.	18.68%
A ADDITIONAL DETAILS ON DATASETS	16	≈ 5 pp.	17.86%
B TRANSCODER TRAINING DETAILS	21	≈ 2 pp.	7.14%
C ADDITIONAL CLASSIFICATION DETAILS	23	≈ 4.21 pp.	15.04%
D USAGE OF LANGUAGE LANGUAGE MODELS	27	≈ 0.1 pp.	0.36%
E LIMITATIONS	27	≈ 1.59 pp.	5.68%

All detected figure and table locations

Main text

• Figure 1 · p. 3
• Table 1 · p. 6
• Table 2 · p. 6
• Figure 2 · p. 7
• Table 3 · p. 7
• Figure 4 · p. 8
• Figure 3 · p. 8
• Table 4 · p. 9

Attached appendix

• Table 5 · p. 16
• Table 6 · p. 17
• Table 7 · p. 18
• Table 8 · p. 18
• Table 9 · p. 19
• Table 10 · p. 19
• Table 11 · p. 20
• Table 12 · p. 20
• Table 13 · p. 20
• Figure 5 · p. 21
• Table 14 · p. 22
• Table 15 · p. 22
• Table 16 · p. 24
• Table 17 · p. 24
• Table 18 · p. 25
• Figure 6 · p. 26
• Figure 7 · p. 26
• Figure 8 · p. 26
• Table 19 · p. 27

Argument and organization

1. Introduction / Problem Formulation and Preliminaries

   Defines a diagnostic scientific instrument and the structural-fingerprint hypothesis.

2. Methodology

   Organizes data generation and four computational stages before evaluation.

3. Experiments

   Progresses from verification to mechanistic interpretation and exploratory intervention.

4. Related Work / Conclusion / Appendices

   Positions the contribution, exposes reproducibility details and bounds the claims.

Problem, claim and contribution

The introduction explicitly chooses mechanistic understanding over production efficiency. It limits experiments to instruction-tuned autoregressive reasoning, explaining why complex search/backtracking models are not the initial target.

§1, p.2 · Main paper

Argument and section order

Three research questions form an evidence ladder: predictive utility, interpretable structural differences and exploratory causal intervention. The last is presented as a case study rather than an aggregate repair benchmark.

§4, p.5; §4.4, pp.8–9 · Main paper

Figures, captions and visual explanation

Figure 1 aligns the four numbered stages with residual-stream diagrams, sparse feature pathways, grouped statistics and classifier output. Table 4 then places before/after traces side by side, highlighting the exact reasoning divergence.

Figure 1, p.3; Table 4, p.9 · Main paper

Tables, comparisons and evidence

Table 1 defines metric directions and flags extreme Boolean label imbalance in its caption. Table 2 keeps poor zero-shot transfer visible, separating domain-specific error signals from a claim of universal verification.

§4.1, p.5; Tables 1–2, p.6 · Main paper

Appendix and reproducibility

Appendix A explains why steps after the first error are discarded and reports both raw annotator agreement and kappa. Appendix C gives probe hyperparameter searches and alternative classifiers, including settings that outperform the main classifier.

Appendices A.3–A.5, pp.17–20; C.2–C.4, pp.23–25 · Main paper

Clarity and writing conventions

The limitations separate compute, aggregate-versus-semantic analysis, one-model generalization and interpretability-tool fidelity. Those qualifications are stronger teaching examples than the occasional rhetoric describing projection plots as definitive computational-integrity regions.

§4.3, p.8; Appendix E, pp.27–28 · Main paper

What to reuse—and what not to infer

Transferable lesson: For an analysis instrument, state the access and compute assumptions early, disclose failed transfer, and distinguish a successful intervention example from a generally validated repair method.

Boundary: CRV analyzes an approximate transcoder replacement, not an exact transparent copy of the original model. Its manual feature interventions are exploratory. Table 15 does not support an unconditional claim that post-step attribution wins nearly every metric; Table 18 likewise shows no universally best classifier.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

PDF pp.1–10, §§1–6 and ethics/reproducibility statements; references pp.10–15

Appendix / supplement

PDF pp.16–28, Appendices A–E: data construction, transcoders, classifiers, additional interventions, disclosure and limitations

Visually inspected PDF pages

3, 6

• official ICLR 2026 proceedings PDF · 28 pages