Writing study: Diffract: Spectral View of LLM Domain Adaptation
← ICML 2026 oral writing corpus
Nikita Borodin, Maria Krylova, Artem Zabolotnyi, Dmitry Aspisov, Egor Shikov, Nikita Tyuplyaev, Oleg Travkin, Roman Alferov, Dmitry Vinichenko · Official ICML 2026 oral record
Figure, table and section ledger
Editorial contribution type: Empirical analysis, diagnosis & auditing (primary).
Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.
- Canonical PDF
- 26 physical pages
- Before references / appendix
- ≈ 8.6 page-equivalents
- Main figures / tables
- 7 / 0
- Attached appendix figures / tables
- 15 / 5
- Attached appendix span
- ≈ 14.91 page-equivalents
Section-by-section lengths and source pages
The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.
| Top-level heading | Starts on PDF page | Estimated span | Share of entire PDF |
|---|---|---|---|
| 1. Introduction | 1 | ≈ 0.98 pp. | 3.77% |
| 2. Methodology | 2 | ≈ 0.33 pp. | 1.27% |
| 3. Experimental Setup | 2 | ≈ 0.75 pp. | 2.88% |
| 4. Main Results | 3 | ≈ 4.44 pp. | 17.08% |
| 5. Related Work | 7 | ≈ 1.02 pp. | 3.92% |
| 6. Discussion and Directions for Further Work | 8 | ≈ 0.71 pp. | 2.73% |
| Impact Statement | 9 | ≈ 0.06 pp. | 0.23% |
| References | 9 | ≈ 2.48 pp. | 9.54% |
| A. Diffract methodology and technical details | 12 | ≈ 2 pp. | 7.69% |
| B. Training and Reproducibility Details | 14 | ≈ 3 pp. | 11.54% |
| C. Additional Experimental Results | 17 | ≈ 7 pp. | 26.92% |
| D. Statistical Robustness | 24 | ≈ 1 pp. | 3.85% |
| E. Key Diffract Visualization Tools | 25 | ≈ 1.91 pp. | 7.35% |
All detected figure and table locations
Main text
Attached appendix
- Table 1 · p. 14
- Table 2 · p. 15
- Table 3 · p. 15
- Figure 7 · p. 17
- Figure 8 · p. 17
- Figure 9 · p. 17
- Figure 10 · p. 18
- Figure 11 · p. 18
- Figure 12 · p. 18
- Figure 13 · p. 19
- Figure 14 · p. 19
- Figure 15 · p. 20
- Figure 16 · p. 20
- Figure 17 · p. 21
- Figure 18 · p. 21
- Figure 19 · p. 22
- Table 4 · p. 22
- Figure 20 · p. 23
- Figure 21 · p. 23
- Table 5 · p. 24
Argument and organization
- Problem and contribution
Connects continual-pretraining changes to both singular magnitudes and directions, separating head-specific behavior from aggregate matrix summaries.
- Argument sequence
Spectral observations lead to head rewinding and delta truncation, then interpolation tests ask whether domain-adapted checkpoints remain connected in quality space.
- Evidence and verification
The appendix separates evaluation bootstrap variation on fixed checkpoints from randomness in head selection; these are not repeated independent training runs.
Problem, claim and contribution
Connects continual-pretraining changes to both singular magnitudes and directions, separating head-specific behavior from aggregate matrix summaries.
Contributions and methodology, p. 2 · Main paper
Argument and section order
Spectral observations lead to head rewinding and delta truncation, then interpolation tests ask whether domain-adapted checkpoints remain connected in quality space.
Sections 4.2-4.5, pp. 4-8 · Main paper
Figures, captions and visual explanation
Aligned head/layer heatmaps compare short and long pretraining with an explicit difference panel; interpolation curves show two task qualities rather than hiding tradeoffs in one average.
Figure 3, p. 5; Figure 6, p. 8; both PNGs inspected · Main paper
Tables, comparisons and evidence
The appendix separates evaluation bootstrap variation on fixed checkpoints from randomness in head selection; these are not repeated independent training runs.
Appendix D and Table 5, p. 24 · Main paper
Appendix and reproducibility
Metric definitions and fit procedures precede extra model/domain results, while visualization documentation explains how distributions and directional agreement can change independently.
Appendix A, pp. 12-13; E, pp. 25-26 · Main paper
Clarity and writing conventions
The interpretation of simultaneous spectral/head changes is explicitly framed as a hypothesis, while model-family scope is restated in the discussion.
Section 4.3, p. 5; Section 6, pp. 8-9 · Main paper
What to reuse—and what not to infer
Transferable lesson: Move from descriptive diagnostics to targeted interventions, and label uncertainty by its actual source rather than using a generic multi-seed claim.
Boundary: Fixed-checkpoint bootstrap errors do not quantify training variance; head redundancy and interpolation findings within OLMo do not establish universal mechanistic laws across model families.
Apply this lesson using the argument and evidence worksheet.
Inspection record and sources
- Checked
- 2026-10-07
- Reading status
- Writing-focused manuscript scan + selected visuals inspected
- Main text
- Writing-focused distributed manuscript scan, not exhaustive technical/proof validation. Sections 1-6, pp. 1-9: spectral diagnostic definitions, pretraining/CPT design, head heterogeneity, rewind/truncation interventions and domain-connectivity analysis.
- Appendix / supplement
- Writing-focused appendix scan; code, data and external assets not executed. Appendices A-E, pp. 12-26: spectral measures/fits, training and code, transplantation and additional domains, statistical-robustness protocol and visualization guidance.
- Visually inspected PDF pages
- 5, 8
This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.