Writing study: Chebyshev Policies and the Mountain Car Problem: Reinforcement Learning for Low-Dimensional Control Tasks

ICML overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICML 2026 oral writing corpus

Stefan Huber, Hannes Unger, Georg Schäfer, Jakob Rehrl · Official ICML 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Architecture & representation (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

23 physical pages

Before references / appendix

≈ 8.74 page-equivalents

Main figures / tables

5 / 2

Attached appendix figures / tables

9 / 8

Attached appendix span

≈ 12.91 page-equivalents

Before references / appendix: ≈ 38%References / intervening material: ≈ 5.87%Attached appendix: ≈ 56.13%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1. Introduction	1	≈ 1.35 pp.	5.87%
2. Analytical Solution to Mountain Car	2	≈ 2.88 pp.	12.52%
3. Optimal Policy for Mountain Car	5	≈ 1 pp.	4.35%
4. Chebyshev Policies	6	≈ 1 pp.	4.35%
5. Evaluation	7	≈ 1.6 pp.	6.96%
6. Conclusion	9	≈ 0.44 pp.	1.91%
Acknowledgment	9	≈ 0.06 pp.	0.26%
Impact Statement	9	≈ 0.08 pp.	0.35%
References	9	≈ 1.35 pp.	5.87%
A. Details of the Analytical Solution to Mountain Car	11	≈ 3 pp.	13.04%
B. Details on Chebyshev Policies	14	≈ 0.38 pp.	1.65%
C. Details on Mountain Car Experiments	14	≈ 5.85 pp.	25.43%
D. Experimental Results on Additional Environments	20	≈ 3.69 pp.	16.04%

All detected figure and table locations

Main text

• Figure 1 · p. 1
• Figure 2 · p. 4
• Figure 3 · p. 4
• Figure 4 · p. 5
• Figure 5 · p. 6
• Table 1 · p. 7
• Table 2 · p. 8

Attached appendix

• Figure 6 · p. 12
• Figure 7 · p. 13
• Table 1 · p. 14
• Figure 8 · p. 15
• Table 3 · p. 16
• Figure 9 · p. 16
• Table 4 · p. 17
• Figure 10 · p. 17
• Figure 11 · p. 17
• Table 5 · p. 19
• Table 6 · p. 20
• Table 7 · p. 20
• Figure 12 · p. 21
• Figure 13 · p. 21
• Figure 14 · p. 22
• Table 8 · p. 23
• Table 9 · p. 23

Argument and organization

1. Problem and contribution

   Uses a precisely specified benchmark dynamics/reward model to replace a solved-task threshold with comparison against an analytical control reference.

2. Argument sequence

   Single- and two-phase trajectories lead to a reference policy, then the new policy family is evaluated through returns, state-space actions and low-dimensional transfer.

3. Evidence and verification

   Simulation and physical-system outcomes are reported separately, while the main text names combinatorial basis growth and the appendix records divergent optimizer runs and best-policy selection.

Problem, claim and contribution

Uses a precisely specified benchmark dynamics/reward model to replace a solved-task threshold with comparison against an analytical control reference.

Sections 1.2-2, p. 2 · Main paper

Argument and section order

Single- and two-phase trajectories lead to a reference policy, then the new policy family is evaluated through returns, state-space actions and low-dimensional transfer.

Sections 2-5, pp. 2-8 · Main paper

Figures, captions and visual explanation

A phase-space diagram distinguishes wall-reset trajectories; return-versus-start-position and action heatmaps expose failure behavior hidden by the average return.

Figure 4, p. 5; Figure 5 and Table 2, p. 8; both PNGs inspected · Main paper

Tables, comparisons and evidence

Simulation and physical-system outcomes are reported separately, while the main text names combinatorial basis growth and the appendix records divergent optimizer runs and best-policy selection.

Limitations, p. 8; Appendix C.2, p. 15; D and Tables 8-9, p. 23 · Main paper

Appendix and reproducibility

The discrete-control comparison acknowledges that numerical optimization need not find a global optimum, and that its evidence is approximate agreement with continuous-time analysis.

Appendix A.5, p. 13 · Main paper

Clarity and writing conventions

Question-led evaluation paragraphs link performance differences to interpretable control strategies, but optimality wording requires the continuous/discrete and numerical-search distinctions.

Section 5, pp. 7-8; Appendix A.5, p. 13 · Main paper

What to reuse—and what not to infer

Transferable lesson: Use an interpretable reference solution to inspect learned behavior, not just to produce one benchmark score, and keep simulation-to-hardware evidence distinct.

Boundary: Universal approximation does not imply efficient high-dimensional scaling; selected best training runs and numerical discrete-time checks are not an unconditional global optimality proof.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-07

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-focused distributed manuscript scan, not exhaustive technical/proof validation. Sections 1-6, pp. 1-9: control problem specification, continuous analytical trajectories, worst-case policy, Chebyshev policy representation and simulated/physical low-dimensional tests.

Appendix / supplement

Writing-focused appendix scan; code, data and external assets not executed. Appendices A-D, pp. 11-23: trajectory proofs, discrete-time numerical checks, polynomial evaluation, training/model controls, dimensionality and physical-system measurements.

Visually inspected PDF pages

5, 8

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Published ICML 2026 proceedings PDF, PMLR volume 306; retrieved 2026-10-07. Public OpenReview decision notes were not independently reverified. · 23 pages