Writing study: MrRoPE: Mixed-radix Rotary Position Embedding

ICLR overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← ICLR 2026 oral writing corpus

Qingyuan Tian, Wenhong Zhu, Xiaoran Liu, Xiaofeng Wang, Rui Wang · Official ICLR 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Architecture & representation design (primary) · Empirical analysis, diagnosis & auditing.

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

16 physical pages

Before references / appendix

≈ 9.44 page-equivalents

Main figures / tables

6 / 3

Attached appendix figures / tables

2 / 2

Attached appendix span

≈ 4.56 page-equivalents

Before references / appendix: ≈ 59%References / intervening material: ≈ 12.5%Attached appendix: ≈ 28.5%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 INTRODUCTION	1	≈ 1.08 pp.	6.75%
2 ROPE AND RADIX THEORY	2	≈ 1.5 pp.	9.38%
3 MRROPE: A UNIFY FRAMEWORK FOR ROPE EXTENSION	4	≈ 2.44 pp.	15.25%
4 EXPERIMENTS	6	≈ 3.31 pp.	20.69%
5 CONCLUSION	9	≈ 0.34 pp.	2.13%
ETHICS STATEMENT	10	≈ 0.12 pp.	0.75%
REPRODUCIBILITY STATEMENT	10	≈ 0.12 pp.	0.75%
REFERENCES	10	≈ 2 pp.	12.5%
A RELATED WORKS	12	≈ 1.73 pp.	10.81%
B DETAILS OF MRROPE METHODOLOGY	14	≈ 1.25 pp.	7.81%
C ADDITIONAL EXPERIMENTS RESULTS	15	≈ 0.37 pp.	2.31%
D USE OF LLMS	15	≈ 0.63 pp.	3.94%
E LIMITATION	16	≈ 0.57 pp.	3.56%

All detected figure and table locations

Main text

• Figure 1 · p. 2
• Figure 2 · p. 3
• Figure 3 · p. 6
• Table 1 · p. 7
• Figure 4 · p. 7
• Table 2 · p. 8
• Table 3 · p. 8
• Figure 5 · p. 9
• Figure 6 · p. 9

Attached appendix

• Figure 7 · p. 14
• Figure 8 · p. 15
• Table 4 · p. 15
• Table 5 · p. 16

Argument and organization

1. Introduction and mixed-radix interpretation

   Recast competing frequency-scaling schemes as choices within one parameterization.

2. Uniform and progressive MrRoPE

   Present two small interventions before examining language modeling, retrieval and downstream tasks.

3. Analysis and supplementary experiments

   Connect empirical context extension to a cosine-sum proxy and expose implementation choices.

Problem, claim and contribution

The contribution is introduced as a common language for existing context-extension methods, making the subsequent uniform/progressive choices interpretable as parameter changes rather than an unrelated architecture.

Sections 1–3, pages 1–6 · Main paper

Argument and section order

The method separates boundary selection, middle-band scaling and attention rescaling. This decomposition helps distinguish the actual new parameterization from retained YaRN implementation components.

Section 3; Appendix B, pages 14–15 · Main paper

Figures, captions and visual explanation

Figure 1 uses aligned frequency bands and radix blocks to translate a numerical encoding into a visual correspondence. Figure 3 then plots cumulative scaling factors, complementing rather than duplicating the symbolic explanation.

Figure 1, page 2; Section 3 and Figure 3 · Main paper

Tables, comparisons and evidence

The evidence moves from perplexity to length-by-depth retrieval heatmaps and then RULER and downstream benchmarks. These answer different questions and should not be collapsed into a single effective-context claim.

Section 4, Tables 1–3, pages 7–9 · Main paper

Appendix and reproducibility

Derivations, hyperparameters and additional model results are separated into named appendices. However, inconsistent model labels and alpha/beta conventions require readers to reconcile the implementation description rather than infer settings from one caption.

Appendices A–C, pages 12–16 · Main paper

Clarity and writing conventions

Some broad superiority wording is stronger than the displayed comparisons: progressive scaling is not the lowest-perplexity choice at every length, including Qwen at 32K. Explicit exceptions would strengthen the argument.

Table 1, page 7; Section 4 discussion · Main paper

What to reuse—and what not to infer

Transferable lesson: Use a shared mathematical parameterization and a matching visual vocabulary to explain a small method change; evaluate distinct consequences with distinct diagnostics instead of treating one metric as a universal proxy.

Boundary: The cosine-sum analysis is a proxy under assumptions, not a guarantee of usable context length. Preserve per-model and per-length exceptions, and resolve appendix notation/model-label inconsistencies before reproducing the method.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-09-19

Reading status

Full text + selected visuals inspected

Main text

Pages 1–10, Sections 1–5, ethics and reproducibility statements

Appendix / supplement

Pages 12–16, Appendices A–E: interpolation derivations, implementation, extra evaluation and limitations

Visually inspected PDF pages

2, 7, 8

• ICLR 2026 final proceedings · 16 pages