Writing study: Unifying Goal-Conditioned RL and Unsupervised Skill Learning via Control-Maximization
← NeurIPS 2026 oral writing corpus
Alireza Modirshanechi, Benjamin Eysenbach, Peter Dayan, Eric Schulz · Official NeurIPS 2026 oral record
Figure, table and section ledger
Editorial contribution type: Theory & formal analysis (primary).
Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.
- Canonical PDF
- 35 physical pages
- Before references / appendix
- ≈ 9.18 page-equivalents
- Main figures / tables
- 5 / 0
- Attached appendix figures / tables
- 3 / 0
- Attached appendix span
- ≈ 21.91 page-equivalents
Source-bound correction: Top-level heading and reference/appendix boundaries checked against source PDF text. Bibliography titles/author initials, numbered contribution/property bullets, diagram steps, equations and TOC entries are not substantive headings. Section spans are page-coordinate estimates, not ink area or prose-only length. This is an author manuscript mirror, not a verified accepted-version template. Source layout uses physical single-column y/page-height, including equation-heavy pages. Appendix span includes its explicit cover/contents/list-of-figures frontmatter; actual lettered subsection boundaries use their substantive starts, not TOC links.
Section-by-section lengths and source pages
The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.
| Top-level heading | Starts on PDF page | Estimated span | Share of entire PDF |
|---|---|---|---|
| 1 Introduction | 1 | ≈ 1.13 pp. | 3.23% |
| 2 Related work | 2 | ≈ 0.88 pp. | 2.51% |
| 3 Notation and preliminaries | 3 | ≈ 1.07 pp. | 3.06% |
| 4 Different GCRL formulations are incompatible | 4 | ≈ 1.94 pp. | 5.54% |
| 5 A precise control-maximization account of GCRL | 6 | ≈ 1.36 pp. | 3.89% |
| 6 Different GCRL formulations need different MISL objectives | 8 | ≈ 1.49 pp. | 4.26% |
| 7 Conclusion | 9 | ≈ 0.51 pp. | 1.46% |
| Acknowledgement | 10 | ≈ 0.09 pp. | 0.26% |
| References | 10 | ≈ 3.91 pp. | 11.17% |
| Contents | 14 | ≈ 1 pp. | 2.86% |
| A Precise statements of the equivalences in Section 4 | 15 | ≈ 3 pp. | 8.57% |
| B Additional statements for sensitivity-control relationships in Section 5 | 18 | ≈ 1 pp. | 2.86% |
| C Additional statements for MI-empowerment relationships in Section 6 | 19 | ≈ 1 pp. | 2.86% |
| D Proofs of the formal statements in the main text | 20 | ≈ 6 pp. | 17.14% |
| E An MI upper bound for the OW formulation | 26 | ≈ 3 pp. | 8.57% |
| F On the attainability of consistency | 29 | ≈ 1 pp. | 2.86% |
| G Extension to general goal distributions | 30 | ≈ 5.91 pp. | 16.89% |
All detected figure and table locations
Attached appendix
Argument and organization
- Problem and contribution
Challenges an apparent unification by first identifying that goal-conditioned RL and skill learning each contain inequivalent objectives; this makes the conditions for any correspondence the central question.
- Argument sequence
Definitions precede an incompatibility result, special-case equivalences, a control-sensitivity account and the mapping to specific mutual-information skill objectives.
- Evidence and verification
Counterexample environments and explicit assumptions carry the evidential burden rather than benchmark tables; the main argument differentiates full ordering equivalence from agreement only at the optimum.
Problem, claim and contribution
Challenges an apparent unification by first identifying that goal-conditioned RL and skill learning each contain inequivalent objectives; this makes the conditions for any correspondence the central question.
Introduction and Section 4, pp. 1–5 · Main paper
Argument and section order
Definitions precede an incompatibility result, special-case equivalences, a control-sensitivity account and the mapping to specific mutual-information skill objectives.
Sections 3–6, pp. 3–9 · Main paper
Figures, captions and visual explanation
Figure 3 presents an equivalence map whose edges name assumptions and proposition references; black edges mean identical policy ordering while blue edges mean shared optimal policies. Figure 2 turns differing goal objectives into a small shared state graph and three optimal-path diagrams under named horizon/discount settings. Everyday descriptions of persistent, exact-time and opportunity-window goals sit beside the graph, then the formal proposition states existence of incompatible policies rather than implying all environments must disagree.
Figure 3, p. 6 (render inspected); Figure 2 and Proposition 4.1, p. 5 (additional rendered inspection) · Main paper
Tables, comparisons and evidence
Counterexample environments and explicit assumptions carry the evidential burden rather than benchmark tables; the main argument differentiates full ordering equivalence from agreement only at the optimum.
Proposition 4.1 and Figures 2–3, pp. 5–6; Figure 7, p. 20 · Main paper
Appendix and reproducibility
Proofs are organized separately from additional assumptions and extensions; a further counterexample shows that maximizing goal sensitivity need not maximize value for every formulation.
Appendices D–G, pp. 20–35; Figure 8, p. 21 · Main paper
Clarity and writing conventions
Section headings state the logical claims, including incompatibility and when it does not matter. The prose translates those claims into an objective-selection warning without pretending to prove empirical training gains.
Sections 4.1–4.2 and 7, pp. 5–6 and 9 · Main paper
What to reuse—and what not to infer
Transferable lesson: For a unification paper, map both correspondences and non-equivalences, label the assumptions on every bridge and supply counterexamples where intuitive equivalence fails.
Boundary: A theoretical correspondence under specified assumptions is not an experimental guarantee for arbitrary skill-learning implementations or environments.
Apply this lesson using the argument and evidence worksheet.
Inspection record and sources
- Checked
- 2026-10-02
- Reading status
- Writing-focused manuscript scan + selected visuals inspected
- Main text
- Writing-oriented scan across manuscript framing, section sequence, figure/table captions, selected argument/evaluation prose and conclusion; not exhaustive technical or proof verification. Sections 1–7, pp. 1–9; references pp. 10–14. Source is the exact public author version listed below, not a verified accepted/camera-ready revision.
- Appendix / supplement
- Writing-oriented appendix/documentation scan: Appendices A–G, pp. 15–35: equivalence conditions, controllability links, mutual-information bounds, main proofs, extra assumptions and generalized goal distributions. Source code, data, model outputs and external video assets were not executed or independently validated.
- Visually inspected PDF pages
- 5, 6
This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.