Writing study: Difficult Examples Hurt Unsupervised Contrastive Learning: A Theoretical Perspective

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Yi-Ge Zhang, Jingyi Cui, Qiran Li, Yisen Wang · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Theory & formal foundations (primary) · Learning & optimization algorithms.

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

31 physical pages

Before references / appendix

≈ 10.11 page-equivalents

Main figures / tables

4 / 6

Attached appendix figures / tables

2 / 5

Attached appendix span

≈ 17.89 page-equivalents

Before references / appendix: ≈ 32.61%References / intervening material: ≈ 9.68%Attached appendix: ≈ 57.71%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Source-bound correction: Caption audit 2026-09-19, 5LMdnUdAoy: rendered physical PDF page 3 confirms Figure 2, the grouped bar chart about excluding mixed difficult examples. Its caption label is merged with adjacent body text and its following caption words are separate bbox lines. Position uses yMin 211.545544 / 792 plus page index 2.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 1.16 pp.	3.74%
2 DIFFICULT EXAMPLES HURT: A MIXING IMAGE EXPERIMENT	2	≈ 0.77 pp.	2.48%
3 THEORETICAL CHARACTERIZATION OF WHY DIFFICULT EXAMPLES HURT	3	≈ 2.17 pp.	7%
4 THEORETICAL CHARACTERIZATION ON ELIMINATING EFFECTS OF	5	≈ 1.75 pp.	5.65%
5 VERIFICATION EXPERIMENTS	7	≈ 3.12 pp.	10.06%
6 CONCLUSION	10	≈ 0.28 pp.	0.9%
ETHICS STATEMENT	10	≈ 0.27 pp.	0.87%
REFERENCES	11	≈ 3 pp.	9.68%
A TECHNICAL APPENDICES AND SUPPLEMENTARY MATERIAL	14	≈ 5.66 pp.	18.26%
B PROOFS	19	≈ 12.09 pp.	39%
C USAGE OF LLM	31	≈ 0.15 pp.	0.48%

All detected figure and table locations

Main text

• Figure 1 · p. 1
• Figure 2 · p. 3
• Figure 3 · p. 4
• Figure 4 · p. 8
• Table 1 · p. 8
• Table 2 · p. 9
• Table 3 · p. 9
• Table 4 · p. 9
• Table 5 · p. 10
• Table 6 · p. 10

Attached appendix

• Figure 5 · p. 16
• Table 7 · p. 17
• Figure 6 · p. 17
• Table 8 · p. 17
• Table 9 · p. 18
• Table 10 · p. 19
• Table 11 · p. 19

Argument and organization

1. Observation → controlled example → graph model

   Motivates a surprising training effect before formalizing assumptions.

2. Removal, margin, temperature

   Converts the analysis into three interventions and matched comparisons.

Problem, claim and contribution

The opening moves from coreset observations to deliberately mixed CIFAR examples, giving the theoretical question an empirical entry point.

Section 1; Figures 1–2, pages 1–3 · Main paper

Argument and section order

Three intervention subsections reuse the same difficult-example definition, so the reader can compare removal, margin, and temperature without learning a new model each time.

Section 4, pages 5–8 · Main paper

Figures, captions and visual explanation

Figure 3 aligns colored sample groups with the corresponding block-affinity matrix; this makes the graph abstraction and its parameters easier to follow.

Figure 3, page 4 · Main paper

Tables, comparisons and evidence

Tables 2 and 3 separately compare applying each change to all examples versus selected examples, followed by a combined-method table; the controls address the selection mechanism directly.

Tables 2–4, page 9 · Main paper

Appendix and reproducibility

The explicit loss formulas reveal that the implementation called removal zeros selected similarity logits; those terms still contribute exp(0) to the denominator. This operational distinction belongs near the main claim.

Appendix A.3, Equation 13, page 15 · Main paper

Clarity and writing conventions

Upper-bound comparisons should be described as bound comparisons, not proof that actual errors are ordered universally; the removal improvement also has a stated parameter condition.

Theorems 4.1–4.3; Appendix B · Main paper

What to reuse—and what not to infer

Transferable lesson: Build a theory-driven paper around one observable phenomenon, then use parallel interventions and matched controls to connect the abstraction to implementation.

Boundary: Spectral-loss assumptions and bound ordering do not establish universal InfoNCE improvement; masking logits is not deleting training samples or removing denominator terms.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

Pages 1–10, Sections 1–6 and ethics statement; references on pages 11–13

Appendix / supplement

Pages 14–31, Appendices A–C, including implementation, extra experiments, proofs, and disclosure

Visually inspected PDF pages

4, 9

• ICLR 2026 final proceedings · 31 pages