Writing study: Mechanistic Data Attribution: Tracing the Training Origins of Interpretable LLM Units

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Jianhui Chen, Yuzhang Luo, Liangming Pan · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Empirical analysis, diagnosis & auditing (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

33 physical pages

Before references / appendix

≈ 9.25 page-equivalents

Main figures / tables

6 / 2

Attached appendix figures / tables

9 / 6

Attached appendix span

≈ 20.91 page-equivalents

Before references / appendix: ≈ 28.03%References / intervening material: ≈ 8.61%Attached appendix: ≈ 63.36%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Source-bound correction: Join wrapped printed heading text using adjacent source lines; original heading start coordinates are unchanged.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 1.46 pp.	4.42%
2. Related Work	2	≈ 0.55 pp.	1.67%
3. Mechanistic Data Attribution Framework	3	≈ 0.97 pp.	2.94%
4. Causal Validation: Data Influence on Mechanistic Emergence	4	≈ 1.45 pp.	4.39%
5. Mechanistic Insights into Induction Head	5	≈ 2.05 pp.	6.21%
6. Mechanistic Data Augmentation	7	≈ 1.52 pp.	4.61%
7. Discussions	9	≈ 0.5 pp.	1.52%
8. Conclusion	9	≈ 0.18 pp.	0.55%
Impact Statement	10	≈ 0.16 pp.	0.48%
Acknowledgment	10	≈ 0.05 pp.	0.15%
References	10	≈ 2.83 pp.	8.58%
A. Theoretical Background	13	≈ 3.3 pp.	10%
B. From the perspective of Information geometry	16	≈ 1.91 pp.	5.79%
C. Induction Attention Score and Formation Time	18	≈ 1.01 pp.	3.06%
D. Detailed Framework Instantiation and Extensions	19	≈ 1.23 pp.	3.73%
E. Detailed Experimental Setup	20	≈ 1.74 pp.	5.27%
F. Mechanism-Localized Attribution and Multi-Head Circuit Evolution	22	≈ 1.3 pp.	3.94%
G. Robustness and Additional Validation	23	≈ 2.9 pp.	8.79%
H. Ablation Study on Insertion Dynamics	26	≈ 0.29 pp.	0.88%
I. Extended Training Dynamics and Window Selection	26	≈ 0.9 pp.	2.73%
J. Full Distribution of Influence Scores	27	≈ 1.41 pp.	4.27%
K. Implementation Details of Mechanistic Data Augmentation Pipeline	29	≈ 1.1 pp.	3.33%
L. Qualitative Inspection of High-Influence Samples	30	≈ 3.81 pp.	11.55%

All detected figure and table locations

Main text

• Figure 1 · p. 1
• Figure 2 · p. 4
• Figure 3 · p. 5
• Table 1 · p. 5
• Figure 4 · p. 6
• Figure 5 · p. 7
• Table 2 · p. 8
• Figure 6 · p. 9

Attached appendix

• Table 3 · p. 20
• Table 4 · p. 21
• Table 5 · p. 21
• Table 6 · p. 22
• Figure 7 · p. 23
• Figure 8 · p. 24
• Figure 9 · p. 24
• Figure 10 · p. 25
• Figure 11 · p. 26
• Figure 12 · p. 27
• Figure 13 · p. 28
• Figure 14 · p. 31
• Figure 15 · p. 31
• Table 7 · p. 32
• Table 8 · p. 33

Argument and organization

1. 1 Introduction; 2 Related Work; 3 MDA Framework

   Connects interpretable units to their training origins and defines component-level attribution.

2. 4 Causal Validation; 5 Mechanistic Insights

   Tests data interventions before studying sample structure, transfer and emergence dynamics.

3. 6 Data Augmentation; 7 Discussions; 8 Conclusion

   Turns attributed patterns into a synthesis pipeline and qualifies the scope of unit-level explanation.

Problem, claim and contribution

The opening frames the missing link as training origins of already-interpretable units, positioning attribution and intervention as connected but separate activities.

Abstract and Figure 1, p.1 · Main paper

Argument and section order

The method first localizes a target unit, then computes influence, then validates removal/augmentation interventions before using the patterns for data generation.

Sections 3-6, pp.3-7 · Main paper

Figures, captions and visual explanation

A vertically staged overview maps unit localization to influence and intervention; the validation grid reuses line colors for intervention type and line styles for random versus targeted selection across model sizes.

Figure 1, p.1; Figure 2, p.4 · Main paper

Tables, comparisons and evidence

The text distinguishes controlled retraining on small Pythia models from qualitative tracing on billion-parameter OLMo models and acknowledges eventual convergence despite altered emergence rates.

Sections 4.2-4.3, p.5 · Main paper

Appendix and reproducibility

The supplement separates influence-function assumptions from implementation, including high-frequency checkpointing and the LLM-driven pattern-to-generator pipeline.

Appendices A.1, E.1 and K, pp.13,20,29 · Main paper

Clarity and writing conventions

The ICL discussion leaves latent confounders possible, and the final discussion distinguishes direct unit attribution from upstream cascading effects and abstract mechanisms from individual units.

Section 5.4, p.7; Section 7, p.9 · Main paper

What to reuse—and what not to infer

Transferable lesson: Keep attribution scores, intervention evidence and downstream applications distinct, and state where scaling changes the strength of validation.

Boundary: The inspected analysis does not independently verify causal necessity/sufficiency or influence approximations; larger-model tracing lacks the same full-retraining validation.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-oriented scan of the main argument through physical p.9, with focused physical-page text inspection on 1, 3, 5, 7, 9. Framing, section progression, mechanism/proof explanation, displays, comparison conditions and conclusions were inspected. This is not exhaustive technical reading, proof verification or reproduction.

Appendix / supplement

Appendix organization and main-text pointers were checked; focused text inspection of physical pages 13, 20, 29. Remaining appendix proofs and details were not exhaustively read. No separately linked supplement inspected.

Visually inspected PDF pages

1, 4

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published proceedings PDF, PMLR volume 306, retrieved 2026-10-07 · 33 pages