Writing study: Benchmarking Empirical Privacy Protection for Adaptations of Large Language Models

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Bartłomiej Marek, Lorenzo Rossi, Vincent Hanke, Xun Wang, Michael Backes, Franziska Boenisch, Adam Dziedzic · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Datasets, benchmarks & measurement resources (primary) · Empirical analysis, diagnosis & auditing.

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

38 physical pages

Before references / appendix

≈ 10.25 page-equivalents

Main figures / tables

6 / 3

Attached appendix figures / tables

8 / 37

Attached appendix span

≈ 19.89 page-equivalents

Before references / appendix: ≈ 26.97%References / intervening material: ≈ 20.68%Attached appendix: ≈ 52.34%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 1.08 pp.	2.84%
2 BACKGROUND AND RELATED WORK	2	≈ 0.95 pp.	2.5%
3 EXPERIMENTAL SETUP	3	≈ 1.82 pp.	4.79%
4 BENCHMARK DESIGN AND EXPERIMENTS	5	≈ 3.02 pp.	7.95%
5 DISCUSSION OF OUR RESULTS	8	≈ 0.72 pp.	1.89%
6 HOLISTIC PRIVACY AUDITS UNDER THE PRETRAIN-ADAPT PARADIGM	9	≈ 1.53 pp.	4.03%
7 CONCLUSIONS	10	≈ 0.39 pp.	1.03%
ACKNOWLEDGEMENTS	11	≈ 0.15 pp.	0.39%
REFERENCES	11	≈ 7.85 pp.	20.66%
A BACKGROUND	19	≈ 1.5 pp.	3.95%
B ADDITIONAL DETAILS ON THE SETUP	20	≈ 2.55 pp.	6.71%
C ADDITIONAL EXPERIMENTAL RESULTS	23	≈ 9.46 pp.	24.89%
D INFLUENCE OF THE ATTACKER’S KNOWLEDGE	32	≈ 0.79 pp.	2.08%
E LOSS VALUES	33	≈ 3.4 pp.	8.95%
F EXPOSURE ESTIMATION	36	≈ 0.95 pp.	2.5%
G MEMORIZATION OF THE PRETRAINED MODEL	37	≈ 0.35 pp.	0.92%
H RMIA HYPERPARAMETERS	38	≈ 0.09 pp.	0.24%
I BROADER IMPACT	38	≈ 0.47 pp.	1.24%
J LIMITATIONS	38	≈ 0.33 pp.	0.87%

All detected figure and table locations

Main text

• Figure 1 · p. 2
• Table 1 · p. 4
• Table 2 · p. 5
• Table 3 · p. 5
• Figure 2 · p. 6
• Figure 3 · p. 7
• Figure 4 · p. 8
• Figure 5 · p. 9
• Figure 6 · p. 9

Attached appendix

• Figure 7 · p. 24
• Table 4 · p. 24
• Table 5 · p. 25
• Table 6 · p. 25
• Table 7 · p. 25
• Table 8 · p. 26
• Table 9 · p. 26
• Table 10 · p. 26
• Table 11 · p. 27
• Table 12 · p. 27
• Table 13 · p. 27
• Table 14 · p. 28
• Table 15 · p. 28
• Table 16 · p. 28
• Table 17 · p. 29
• Table 18 · p. 29
• Table 19 · p. 29
• Table 20 · p. 30
• Table 21 · p. 30
• Table 22 · p. 30
• Table 23 · p. 31
• Figure 8 · p. 31
• Table 24 · p. 31
• Figure 9 · p. 32
• Figure 10 · p. 32
• Figure 11 · p. 33
• Table 25 · p. 33
• Table 26 · p. 33
• Figure 12 · p. 34
• Table 27 · p. 35
• Table 28 · p. 35
• Table 29 · p. 35
• Table 30 · p. 35
• Table 31 · p. 35
• Table 32 · p. 35
• Table 33 · p. 35
• Table 34 · p. 36
• Table 35 · p. 36
• Table 36 · p. 36
• Table 37 · p. 36
• Table 38 · p. 36
• Table 39 · p. 37
• Figure 13 · p. 37
• Table 40 · p. 37
• Figure 14 · p. 38

Argument and organization

1. Benchmark setup and six research questions

   Organizes empirical privacy comparisons by data relationship, adaptation, extraction, attacker knowledge, pretraining leakage, and utility.

2. Holistic auditing framework

   Separates four pipeline stages with explicit adversary games after the empirical findings.

Problem, claim and contribution

A single practical question is decomposed into six numbered research questions. Repeated Motivation, Summary of Findings, and Detailed Results paragraphs provide a reusable pattern for a large experimental benchmark.

Sections 4.1–4.6, pp. 5–8 · Main paper

Argument and section order

The four audit stages explicitly vary which models and membership variables the attacker observes. This prevents pretraining leakage from being silently treated as the same estimand as adaptation membership.

Section 6.1, pp. 9–10 · Main paper

Figures, captions and visual explanation

The opening schematic identifies data distributions, adaptation, and two audit types. Later side-by-side diagrams distinguish stage-specific and joint audits, preparing readers for changes highlighted in the adversarial game.

Figure 1, p. 2; Figures 5–6, p. 9 · Main paper

Tables, comparisons and evidence

Tables group privacy budgets within datasets and separate attacker assumptions into row blocks. This helps compare like with like, although the exceptionally small type in the IID table makes detailed inspection difficult.

Tables 2–3, p. 5 · Main paper

Appendix and reproducibility

The appendix supplies attack pseudocode, canary construction, hyperparameter ranges, additional model families, and utility tables. Reporting losses alongside attack scores makes imperfect utility matching visible.

Appendix B, pp. 20–22; Tables 26–39, pp. 33–37 · Main paper

Clarity and writing conventions

A finding reverses direction in the discussion: Section 4.3 calls prefix tuning most vulnerable to extraction, while Section 5 calls it least vulnerable. Distinguishing adaptation canaries from previously memorized pretraining strings would avoid that contradiction.

Sections 4.3 and 5, pp. 6 and 8; Tables 22–23, pp. 30–31 · Main paper

What to reuse—and what not to infer

Transferable lesson: Use research questions as navigation, but preserve the exact privacy unit, attacker information, and target dataset whenever summarizing a finding.

Boundary: The claimed sentence-level protection is implemented on concatenated 256-token chunks. Empirical attack AUC is not a proof that DP is broken or a complete privacy guarantee; reproducing the accountant, neighboring relation, and attack sampling would be necessary before stronger conclusions. The broad four-stage framework is not fully instantiated.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

Pages 1–10, Sections 1–7

Appendix / supplement

Pages 19–38, appendices A–J: attacks, protocols, all supplementary result tables, utility, exposure estimation, and limitations

Visually inspected PDF pages

2, 5, 9

• ICLR 2026 final proceedings · 38 pages