Writing study: Huxley-Gödel Machine: Human-Level Coding Agent Development by an Approximation of the Optimal Self-Improving Machine

← ICLR 2026 oral writing corpus

Wenyi Wang, Piotr Piękos, Li Nanbo, Firas Laakom, Yimeng Chen, Mateusz Ostaszewski, Mingchen Zhuge, Jürgen Schmidhuber · Official ICLR 2026 oral record

Figure, table and section ledger

Editorial contribution type: Learning & optimization algorithms (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF: 31 physical pages
Before references / appendix: ≈ 10.65 page-equivalents
Main figures / tables: 1 / 3
Attached appendix figures / tables: 0 / 1
Attached appendix span: ≈ 16.89 page-equivalents

Before references / appendix: ≈ 34.35%References / intervening material: ≈ 11.16%Attached appendix: ≈ 54.48%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	2	≈ 1.08 pp.	3.48%
2 SELF-IMPROVING AS TREE-SEARCH	3	≈ 0.57 pp.	1.84%
3 HUXLEY-GÖDEL MACHINE	3	≈ 3.11 pp.	10.03%
4 EXPERIMENTAL RESULTS	6	≈ 2.83 pp.	9.13%
5 RELATED WORKS	9	≈ 0.84 pp.	2.71%
6 CONCLUSION	10	≈ 0.7 pp.	2.26%
ETHICS STATEMENT	11	≈ 0.2 pp.	0.65%
REPRODUCIBILITY STATEMENT	11	≈ 0.2 pp.	0.65%
REFERENCES	11	≈ 3.46 pp.	11.16%
A GÖDEL MACHINE WITH CMP ORACLE	15	≈ 3 pp.	9.68%
B ALGORITHM	18	≈ 1 pp.	3.23%
C EXPERIMENTAL DETAILS	19	≈ 1 pp.	3.23%
D EMPIRICAL CMP AND ITS ESTIMATION	20	≈ 1 pp.	3.23%
E BASELINES	21	≈ 1 pp.	3.23%
F DISCOVERED AGENTS	22	≈ 9.89 pp.	31.9%

All detected figure and table locations

Main text

Attached appendix

Table 4 · p. 21

Argument and organization

Introduction / Self-improving as tree-search
Names a surrogate-objective mismatch and formalizes the development process.
Huxley–Gödel Machine
Separates idealized oracle reasoning from practical clade statistics and scheduling.
Experiments
Tests estimator correlation, discovery efficiency and cross-dataset/backbone transfer.
Appendices A–F
Provides formal assumptions, asynchronous caveats, setup and concrete discovered artifacts.

Problem, claim and contribution

The paper gives a precise name to the gap between current benchmark performance and future self-improvement potential. It then defines separate expansion, evaluation and scheduling decisions, making the proposed optimization target easier to distinguish from task accuracy.

§§1–2, pp.2–3 · Main paper

Argument and section order

A boxed assumption list precedes the oracle theorem. The practical algorithm follows separately, with clade-aggregated success counts and Thompson sampling rather than access to the theorem's true future-utility oracle.

§§3.1–3.2, pp.4–6 · Main paper

Figures, captions and visual explanation

Tables 1–3 use three compact comparisons for different claims: weighted/unweighted correlation, accuracy plus allocated CPU-hours, and filtered/standard transfer. This prevents one headline score from standing in for estimator quality, efficiency and generalization simultaneously.

Tables 1–3, pp.7–9 · Main paper

Tables, comparisons and evidence

The correlation analysis removes the root and the descendant branch containing the empirical maximum to reduce direct target leakage. The transfer study separately excludes 93 overlapping SWE-bench tasks and matches the smaller backbone in one comparison.

§4.1, p.7; §4.3.2, p.9; Appendix D, p.20 · Main paper

Appendix and reproducibility

The implementation appendix admits asynchronous completion bias toward agents whose easy evaluations finish early. Experimental details distinguish one-hour and five-hour execution limits and disclose total spending, qualifying CPU-hour efficiency as only one resource measure.

Appendices B–C, pp.18–19 · Main paper

Clarity and writing conventions

The long code listings make discovered modifications inspectable, but their interpretation needs restraint: parts merely log proposed fixes. Describing such snippets as autonomous higher-order improvement is stronger than demonstrating that the edits caused benchmark gains.

Appendix F, Listings 1–2, pp.22–31 · Main paper

What to reuse—and what not to infer

Transferable lesson: Define the surrogate-target mismatch, test the surrogate without direct target leakage, and separate theorem assumptions, heuristic implementation and transfer evidence into distinct claims.

Boundary: Human-level here means a dated benchmark comparison against human-engineered agents, not human programmers in general. The oracle theorem excludes proof-search cost and does not prove optimality of the practical estimator. Allocated CPU-hours exclude a full accounting of API/GPU cost. SICA stops early, and small transfer differences lack uncertainty intervals; current leaderboard supremacy is not established by the historical paper.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked: 2026-09-19
Reading status: Full text + selected visuals inspected
Main text: PDF pp.1–10, §§1–6; statements and references pp.11–14
Appendix / supplement: PDF pp.15–31, Appendices A–F, including oracle formalization, pseudocode, cost, estimator definition and both complete patch listings
Visually inspected PDF pages: 7, 8, 9

official ICLR 2026 proceedings PDF · 31 pages