Writing study: Information Flow Reveals When to Trust Language Models

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Rui Xu, Yi Chen, Jiujiu Chen, Sihong Xie · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Empirical analysis, diagnosis & auditing (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

20 physical pages

Before references / appendix

≈ 9.04 page-equivalents

Main figures / tables

6 / 2

Attached appendix figures / tables

1 / 11

Attached appendix span

≈ 7.91 page-equivalents

Before references / appendix: ≈ 45.2%References / intervening material: ≈ 15.25%Attached appendix: ≈ 39.55%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 0.99 pp.	4.95%
2. Related work	2	≈ 0.55 pp.	2.75%
3. Background	2	≈ 0.89 pp.	4.45%
4. Method	3	≈ 3.48 pp.	17.4%
5. Experiment	7	≈ 0.61 pp.	3.05%
6. Discussion	7	≈ 1.89 pp.	9.45%
7. Conclusion	9	≈ 0.1 pp.	0.5%
Impact Statement	9	≈ 0.05 pp.	0.25%
Acknowledgement	9	≈ 0.15 pp.	0.75%
References	10	≈ 3.04 pp.	15.2%
A. Dominance of self-contributions via residual connection	13	≈ 0.24 pp.	1.2%
B. Computation of rank-biased overlap (RBO)	13	≈ 0.39 pp.	1.95%
C. KL Divergence for discrete probability distributions	13	≈ 0.75 pp.	3.75%
D. Multi-level granularity	14	≈ 0.32 pp.	1.6%
E. Inference Setup and Correctness Evaluation	14	≈ 0.85 pp.	4.25%
F. Stability Analysis of UQ Methods	15	≈ 0.2 pp.	1%
G. Discriminative ability	15	≈ 1.65 pp.	8.25%
H. Computational cost and applicability	17	≈ 0.59 pp.	2.95%
I. Performance on calibration metrics	18	≈ 0.38 pp.	1.9%
J. Ranker model bias measurement	18	≈ 1.13 pp.	5.65%
K. Effect of post-hoc calibration	19	≈ 1.41 pp.	7.05%

All detected figure and table locations

Main text

• Figure 1 · p. 2
• Figure 2 · p. 3
• Figure 3 · p. 3
• Figure 4 · p. 4
• Figure 5 · p. 5
• Table 1 · p. 8
• Table 2 · p. 9
• Figure 6 · p. 9

Attached appendix

• Table 3 · p. 14
• Table 4 · p. 16
• Table 5 · p. 16
• Table 6 · p. 16
• Figure 7 · p. 17
• Table 7 · p. 18
• Table 8 · p. 18
• Table 9 · p. 19
• Table 10 · p. 19
• Table 11 · p. 20
• Table 12 · p. 20
• Table 13 · p. 20

Argument and organization

1. 1 Introduction; 2 Related Work; 3 Background

   Locates the reliability problem in contextual grounding and defines layer-wise contributions.

2. 4 Method

   Progresses from emergence and path aggregation to context slicing, external relevance and confidence aggregation.

3. 5 Experiment; 6 Discussion; 7 Conclusion

   Separates discrimination results from transfer, intervention, ranker bias and calibration controls.

Problem, claim and contribution

The general trust headline is operationalized as uncertainty estimation for retrieved-context question answering, not universal model reliability.

Abstract, p.1; setup, p.7 · Main paper

Argument and section order

The method explains two complementary flow views with a worked path example and analogy before combining them into a supervised calibrator.

Sections 4.2-4.5, pp.5-7 · Main paper

Figures, captions and visual explanation

The path schematic marks admissible and invalid routes using the same colors as the matrix factors; paired ablation bars distinguish removing high-ranked from low-ranked tokens.

Figure 5, p.5; Figure 6, p.9 · Main paper

Tables, comparisons and evidence

The authors test token removal on one hundred initially correct examples, and separately acknowledge that ranker-derived relevance is an estimated reference rather than gold annotation.

Sections 6.2-6.3, p.8 · Main paper

Appendix and reproducibility

Calibration error, ranker verification and quadratic-in-length memory requirements are reported separately from the main AUROC/AUPRC ranking.

Appendices H-J, pp.17-18 · Main paper

Clarity and writing conventions

The results explicitly retain non-winning cells, but causal-faithfulness language should be limited to the tested input-removal intervention and judged correctness protocol.

Section 5.2, p.7; Section 6.2, p.8 · Main paper

What to reuse—and what not to infer

Transferable lesson: Distinguish discrimination, probability calibration, attribution intervention and external-reference quality instead of treating one reliability score as all four.

Boundary: The method requires internal model access, labeled calibration data and an external ranker; token removal does not establish complete causal faithfulness or trust outside the tested RAG setting.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-oriented scan of the main argument through physical p.9, with focused physical-page text inspection on 1, 3, 5, 6, 7, 8, 9. Framing, section progression, mechanism/proof explanation, displays, comparison conditions and conclusions were inspected. This is not exhaustive technical reading, proof verification or reproduction.

Appendix / supplement

Appendix organization and main-text pointers were checked; focused text inspection of physical pages 13, 17, 18. Remaining appendix proofs and details were not exhaustively read. No separately linked supplement inspected.

Visually inspected PDF pages

5, 9

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published proceedings PDF, PMLR volume 306, retrieved 2026-10-07 · 20 pages