Autoregressive language modelling
Reasoning methods alter prompts, samples, search, or training over token sequences.
LLM reasoning and verification
A fluent chain of thought is not proof of a faithful process. This chapter keeps answer quality, process validity, verifier design, compute, and contamination apart.
21 maintained papers · literature reviewed through 2 October 2026
Prompted rationales, sampled consistency, rationale bootstrapping, search over thoughts, reinforcement learning, and test-time scaling can improve task scores through different mechanisms. A visible chain may be incorrect or unfaithful; answer accuracy, process validity, verifier independence, compute, and contamination require separate evidence.
Prerequisite contract
Evaluation and contamination control
Benchmarks and judges can reward memorization or formatting.
How the field developed
2022–2024
Prompted, bootstrapped, supervised, and distilled rationales
Chain-of-thought elicits intermediate text; STaR iteratively trains on rationales that lead to accepted answers; process supervision rewards correct intermediate steps; Distilling Step-by-Step transfers teacher rationales and labels into smaller task models.
Key papers in this section
Order Primary paper · authors · year · venue/status Core idea and why to read it Official resources 01.01 Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Chain-of-Thought prompting augments each few-shot example from an input–answer pair to an input–intermediate reasoning–answer triple. A sufficiently large off-the-shelf language model then imitates that format for a new problem, generates a variable-length natural-language chain, and only afterward emits the final answer. No weights are updated. The paper's central empirical finding is conditional: intermediate reasoning substantially improves several multi-step arithmetic, commonsense, and symbolic tasks at roughly hundred-billion-parameter scale, while smaller models often generate fluent but illogical chains and can do worse than standard prompting. The core mechanism is reasoning tokens before the answer, elicited at inference time. — 01.02 STaR: Bootstrapping Reasoning With Reasoning STaR (Self-Taught Reasoner) bootstraps explicit rationales from a small few-shot rationale prompt and a large answer-labeled dataset. Each round generates rationale–answer pairs, keeps those whose final answer is correct, attempts failed questions again with the correct answer supplied as a hint, removes that hint from accepted training examples, and fine-tunes a fresh copy of the base model on the resulting rationales. Official code 01.03 Let’s Verify Step by Step Let's Verify Step by Step compares reward models trained on only final correctness with reward models trained to judge every reasoning step. A fixed generator produces newline-delimited MATH solutions; an outcome reward model scores final correctness, while a process reward model predicts positive, negative, or neutral at each step and multiplies step-correctness probabilities into a solution score. Both are evaluated by selecting the highest-scoring member of a uniform candidate pool, not by reinforcement-learning the generator. The strongest human-labeled PRM solves 78.2 percent of the paper's held-out MATH subset at Best-of-1860, versus 72.4 for its ORM and 69.6 for majority vote. The memory line is supervise where reasoning first fails. PRM800K dataset 01.04 Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes Distilling Step-by-Step asks a large language model to annotate examples with both a task label and a natural-language rationale, then trains a much smaller model on two outputs. One prefixed task predicts the label; another predicts the rationale. The rationale supplies intermediate task knowledge during training, but the compact model only needs to generate the label at inference. Official code Work: Compare direct answers, outcome supervision, process supervision, rationale bootstrapping, and rationale distillation with identical examples and scoring.
Exit evidence: Separate answer gain, step validity, teacher privilege, verifier dependence, and training-selection bias.
2023
Sampling and deliberate search
Self-consistency aggregates sampled paths; Tree of Thoughts explicitly proposes, scores, and searches intermediate states.
Key papers in this section
Order Primary paper · authors · year · venue/status Core idea and why to read it Official resources 02.01 Self-Consistency Improves Chain of Thought Reasoning in Language Models Self-consistency is an inference-only decoder for chain-of-thought prompting: sample many independent reasoning paths, extract each path's final answer, and return the answer supported by the largest number of paths. It does not try to identify one intrinsically best rationale. Instead, it treats the rationale as a latent route and marginalizes routes at the answer level. The paper's hypothesis is asymmetric: correct reasoning may take diverse forms yet converge on the same answer, whereas independent mistakes are less likely to agree. The memory line is sample paths, vote on answers. — 02.02 Tree of Thoughts: Deliberate Problem Solving with Large Language Models Tree of Thoughts (ToT) wraps a language model in explicit search over coherent intermediate text units called thoughts. A task defines how to decompose states, how the LM proposes candidate thoughts, how it evaluates states by values or votes, and how breadth-first or depth-first search selects, prunes, and backtracks. The LM supplies both proposals and heuristics; the controller supplies deliberate exploration. Official code Work: Match token or wall-clock budget while varying samples, branching, scoring, and stopping.
Exit evidence: Plot quality versus compute and verifier calls.
2024–2025
Mathematical pretraining, reasoning training, and test-time scale
DeepSeekMath combines a curated mathematical corpus with code-informed pretraining and GRPO; DeepSeek-R1 emphasizes RL-driven reasoning behaviour; test-time scaling studies inference allocation and selection.
Key papers in this section
Order Primary paper · authors · year · venue/status Core idea and why to read it Official resources 03.01 DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models DeepSeekMath builds a 7B mathematical reasoner through three linked stages: continue pretraining a code-capable base model on 120 billion math-focused tokens, supervise it on 776 thousand reasoning and tool-use examples, then reinforce correct answers with Group Relative Policy Optimization. GRPO samples a group of answers to each question and normalizes their rewards within the group, eliminating PPO's separately trained critic model. Official models and implementation 03.02 DeepSeek-R1 Incentivizes Reasoning in LLMs through Reinforcement Learning DeepSeek-R1 tests whether long-form reasoning can emerge from reinforcement learning driven chiefly by verifiable final-answer rewards rather than human-written chains of thought. R1-Zero applies Group Relative Policy Optimization (GRPO) directly to DeepSeek-V3 Base with accuracy and format rewards; it learns longer reflection, verification, and alternative-search behavior but remains hard to read and mixes languages. Final R1 wraps that experiment in a multistage pipeline—cold-start supervised data, reasoning RL, rejection-sampled supervised training, and broad preference RL—to retain reasoning while improving usability. The central distinction is R1-Zero discovers; R1 curates and aligns. — 03.03 Scaling LLM Test-Time Compute Optimally Can Be More Effective than Scaling Model Parameters This paper argues that more inference compute is useful only when allocated through the right mechanism for the particular problem. With a PaLM 2-S* math model, it studies two axes: search against a process reward model, and a revision model that changes the proposal distribution by conditioning on earlier attempts. Beam search, best-of-N, parallel sampling, and sequential revision have different scaling behavior across model-relative difficulty bins. A compute-optimal policy first estimates difficulty, then chooses the search algorithm or sequential-to-parallel ratio for the available budget. It reaches or nearly reaches best-of-N with fourfold less counted compute in selected comparisons. The memory line is difficulty chooses how to think longer. — Work: Evaluate held-out, contamination-resistant tasks while separating corpus, supervised data, RL objective, sampling, and selection.
Exit evidence: State what comes from domain data, training, sampling, search, or external verification.
2026
Iterative, verifier, causal, and visual reasoning
The route adds iterative value refinement, verifier-gated distillation, causal ToM steering, expert rule application, hidden-state reranking, multimodal CoT limits, and rubric-based visual rewards; benchmark success is not process faithfulness.
Key papers in this section
Order Primary paper · authors · year · venue/status Core idea and why to read it Official resources 04.01 Evolutionary Guided Decoding: Iterative Value Refinement for LLMsPeer-reviewed ACL 2026 long paper and Outstanding Paper award recipient. Evolutionary Guided Decoding makes the value function evolve with the policy it guides. It samples multiple continuations, uses a task reward to label every prefix with its trajectory outcome, fits a prefix-value model, then decodes a stronger policy by multiplying base next-token probabilities with exponentiated predicted value. The improved policy generates the next training generation, reducing the stale-distribution problem that limits one-shot value-guided search. Earlier author manuscript 04.02 On-policy Distillation with Verifiable RewardarXiv v1 preprint submitted 25 August 2026; evaluation is limited to the reported verifiable reasoning tasks. Core mechanism: verifier-gated token credit. The student samples its own reasoning trajectories, a teacher evaluates token-level preferences along those trajectories, and a verifiable final reward gates the sign of token updates so incorrect solutions are not reinforced as if they were correct. This reading boundary matters: the method provides correctness-aware credit for tasks with programmatic verification, not a general solution for open-ended truthfulness or unverifiable preferences. Official implementation 04.03 CoSToM: Causal-oriented Steering for Intrinsic Theory-of-Mind Alignment in Large Language ModelsPeer-reviewed ACL 2026 long paper and Best Theme Paper award recipient. CoSToM turns a mechanistic probe into a training interface for Theory-of-Mind behavior. One copy of an LLM encodes a dialogue, a frozen second copy receives a patched intermediate activation and answers belief–desire–intention questions, and the resulting loss is backpropagated through the patch into shallow LoRA adapters of the encoder. The tuned encoder is then used alone to generate dialogue whose behavior is intended to reflect the inferred mental state. Author manuscript · Official code 04.04 HSCodeComp: A Realistic and Expert-level Agent Benchmark for Hierarchical Rule ApplicationPeer-reviewed ACL 2026 long paper and Best Resource Paper award recipient; later label-audit concerns should accompany benchmark use. HSCodeComp tests an agent on hierarchical tariff classification where a product description must be converted into an HS code by gathering evidence and applying nested legal and domain rules. The benchmark contains 632 products across 32 categories, labeled with the involvement of 26 tariff experts. Unlike ordinary multiple choice, the agent must sustain a reasoning path through product attributes, rule exceptions, and successive code levels before committing to one classification. Official benchmark repository · Official dataset · Independent primary label-audit manuscript 04.05 Your Reasoning Model is Secretly a Reward Model - Optimization-Free Verification from ExperiencePeer-reviewed ACL 2026 long paper. CLUE uses the reasoning model's own hidden-state movement as a lightweight correctness signal. For each explicit reasoning trace, it subtracts the activation at the start of the reasoning span from the activation at the end, pools the resulting deltas across selected layers, and compares the vector with centroids built from labeled successful and failed experiences. No verifier network is optimized: nearest-centroid geometry supplies a correctness score for filtering or reranking candidates. — 04.06 Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot DoPeer-reviewed ACL 2026 long paper. This study shows that multimodal chain-of-thought is not a uniformly helpful switch. Across 12 tasks, 14 non-reasoning models and 8 reasoning models, longer deliberation helps mathematics, science, and multi-image reasoning more consistently than direct perception tasks. Grounding and counting can stagnate or worsen, while later reflection often becomes less visually engaged. The practical conclusion is to allocate heavy reasoning selectively, not equate more tokens with more visual understanding. Author manuscript 04.07 V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement LearningarXiv preprint first submitted 26 August 2026; not peer reviewed as of the audit date. V-Rubrics replaces one answer-level VLM reward with atomic criteria for visual faithfulness, reasoning consistency, and instruction following. A judge checks which rubric items are satisfied and assigns component and prefix credit, giving GRPO a denser signal than final-answer acceptance. V-Rubrics 50K contains 50,248 records aggregated from 17 sources and supplies image-grounded questions, answers, and generated rubrics for supervised initialization and reinforcement learning. Official project · Official code and data Work: Run a factorial comparison of iterative value refinement, verifier-gated distillation, activation steering, expert rule execution, hidden-state reranking, multimodal chain-of-thought, and decomposed visual rewards with matched sampling and judge budgets.
Exit evidence: Report final correctness, process validity, verifier/judge errors, faithfulness interventions, calibration, number of generations, fitting or steering cost, and failure strata separately.
2026
Anchor-based cross-tokenizer reasoning transfer
ACTD aligns mapped top-k teacher tokens and suppresses unaligned residual probability mass for math-reasoning students with different tokenizers.
Key papers in this section
Order Primary paper · authors · year · venue/status Core idea and why to read it Official resources 05.01 ACTD: Anchor-Based Cross-Tokenizer Distillation with Residual RegularizationThe arXiv record reports acceptance to the EMNLP 2026 main conference. No final ACL Anthology proceedings record was available at the 2026-09-02 audit, so methods and results remain pinned to arXiv v1 submitted 30 August 2026. ACTD distills reasoning models across different tokenizers by mapping teacher top-k tokens into student-space anchors. An anchor L1 term matches probability on aligned tokens, while residual regularization constrains the student's probability mass outside those anchors. With multiple teachers, ACTD computes a complete teacher-specific loss for each teacher and averages those losses; it does not average anchor signals or teacher distributions first. Official implementation Work: Inspect character-boundary mapping, anchor coverage, unmapped mass, teacher diversity, and free-running pass rates.
Exit evidence: Report mapping errors and reasoning performance across model families, tasks, and sampling budgets.
2025–2026
Reasoning budgets, executable outputs and scientific edits
Test-time scaling studies how extra reasoning changes perception and mathematics; EmbedCopilot checks whether plausible embedded-system answers actually execute; GRADE requires domain-correct and readable image edits. Together they motivate measuring compute, implementation success and constraint satisfaction rather than judging prose alone.
Key papers in this section
Order Primary paper · authors · year · venue/status Core idea and why to read it Official resources 06.01 On Test-Time Scaling for Vision-Language ModelsECCV 2026 Spotlight verified in the official Vision-Language Models & Visual Foundation Models session; manuscript #4968 checked on 2026-09-12. Publisher version of record was not independently verified. This empirical study tests when extra inference computation helps vision-language models. Nine prompting and sampling strategies are compared across thirteen instruction-tuned models. The main distinction is task-dependent: more reasoning can help mathematical or logical questions while harming perception-focused judgments. It is not a new trained reasoning model. ECCV Spotlight program · Conference manuscript Work: What external test can falsify a persuasive explanation: a perception answer, a running circuit, or a scientifically valid image?
Exit evidence: More tokens can reduce perceptual accuracy. Judge scores and physical execution use different protocols; GRADE strict accuracy is a conjunction of three maxima, not its mean reasoning score.
2026
Token-level direction and teacher consensus
BinaryOPD tests the sign of sampled-token teacher/student probability differences. C-MOPD resolves agreement and disagreement between task-specialist teachers rather than blindly averaging their signals.
Key papers in this section
Order Primary paper · authors · year · venue/status Core idea and why to read it Official resources 07.01 Do We Really Need KL Divergence for On-Policy Distillation of Large Language Models?Preprint checked 2026-10-02. Empirical results concern autoregressive math and code distillation; the title is not a general theorem eliminating KL from all distillation. BinaryOPD replaces the magnitude of a teacher-student log-probability reward with its sign. The paper then uses high-disagreement token analysis to motivate C-MOPD, which combines teachers while declining strongly conflicting updates. Versioned full text · Author implementation Work: Which disagreement carries useful information, and when should conflicting teachers produce no update?
Exit evidence: Tested math/code token objectives are not a theorem that KL magnitudes or continuous feature residuals can always be discarded. Multiple teacher queries remain a training cost.
2026
Distillation changes capabilities, not only average scores
Distilled Pretraining separates access to teacher data from the effect of soft targets. It contrasts context copying with multi-sample solution coverage and tests entropy-based token routing. A student can improve on pass@k yet become less responsive to supplied context.
Key papers in this section
Order Primary paper · authors · year · venue/status Core idea and why to read it Official resources 08.01 Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scaling Distilled pretraining transfers a teacher’s full next-token distribution, not only sampled text. The study finds a capability trade-off: soft targets can improve multi-sample solution coverage while weakening context-copying behavior in a matched-data regime. Entropy-based token routing partially mitigates the latter rather than making distillation uniformly beneficial. Proceedings PDF · arXiv Work: Which capability benefits from teacher smoothing, and which needs the original hard target?
Exit evidence: Matched token counts do not match total teacher–student compute. Context-copying experiments and pass@k are separate endpoints; routing results are not a universal recipe or demonstrated pathology transfer.
2026
Route additional correction by teacher–student disagreement
DrOPD preserves reverse KL at every student-generated position and adds forward KL at the batch’s highest-JS positions. The candidate union retains tokens preferred by either model. Read it after on-policy KD to distinguish uncertainty, disagreement and teacher-supported modes the student under-covers.
Key papers in this section
Order Primary paper · authors · year · venue/status Core idea and why to read it Official resources 09.01 Divergence-Routed On-Policy Distillation for Language ModelsReview pins arXiv v1 (14 October 2026). The manuscript says under review at ICLR 2027; this is not acceptance. DrOPD changes where an additional teacher signal is applied. It keeps reverse-KL supervision on every student-generated response position and adds forward KL at the highest-disagreement positions. The motivation is under-coverage: reverse KL can barely push up a candidate to which the student already assigns almost no mass. This is a training-objective intervention, not a new language-model architecture. Version 1 paper and appendices Work: Which tokens need extra teacher-weighted correction, and is their teacher target reliable?
Exit evidence: Math-only preprint, two related Qwen pairs. Top-k approximate losses, eight-response mean accuracy and logged training-step cost are separate evidence contracts; no demonstrated WSI transfer.
Research branches
Reasoning elicitation
Does intermediate text improve answers, expose process, or merely add compute?
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models · STaR: Bootstrapping Reasoning With Reasoning
Inference-time search
Who proposes, scores, verifies, and stops candidate paths?
Self-Consistency Improves Chain of Thought Reasoning in Language Models · Tree of Thoughts: Deliberate Problem Solving with Large Language Models
Learned reasoning and verification
Which data and objective teach reasoning, and is the verifier independent, calibrated, and hard to game?
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models · DeepSeek-R1 Incentivizes Reasoning in LLMs through Reinforcement Learning · Scaling LLM Test-Time Compute Optimally Can Be More Effective than Scaling Model Parameters
Value-guided, verifier-gated, causal, rule-based, and visual reasoning
Which component proposes, scores, verifies, steers, or rewards a reasoning path, and is its success evidence about the path or only the final answer?
Evolutionary Guided Decoding: Iterative Value Refinement for LLMs · On-policy Distillation with Verifiable Reward · CoSToM: Causal-oriented Steering for Intrinsic Theory-of-Mind Alignment in Large Language Models · HSCodeComp: A Realistic and Expert-level Agent Benchmark for Hierarchical Rule Application · Your Reasoning Model is Secretly a Reward Model - Optimization-Free Verification from Experience · Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Do · V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning
Tokenizer-mismatched reasoning KD
Which teacher probabilities remain meaningful after vocabulary alignment?
ACTD: Anchor-Based Cross-Tokenizer Distillation with Residual Regularization
Reasoning budgets, executable outputs and scientific edits
What external test can falsify a persuasive explanation: a perception answer, a running circuit, or a scientifically valid image?
Token-level direction and teacher consensus
Which disagreement carries useful information, and when should conflicting teachers produce no update?
Do We Really Need KL Divergence for On-Policy Distillation of Large Language Models?
Distillation changes capabilities, not only average scores
Which capability benefits from teacher smoothing, and which needs the original hard target?
Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scaling
Route additional correction by teacher–student disagreement
Which tokens need extra teacher-weighted correction, and is their teacher target reliable?
Divergence-Routed On-Policy Distillation for Language Models
Comparison table
| Family or stage | Scientific distinction | Main risk | Anchors |
|---|---|---|---|
| Prompted trace | Changes the generated token trajectory without parameter updates. | Trace fluency can be mistaken for faithfulness. | 2022 |
| Sampling / search | Spends inference compute on multiple candidate paths. | Unequal compute or oracle selection inflates gains. | 2023 · 2023 |
| Reasoning-trained model | Changes parameters using domain corpora, rationales, rewards, or outcome signals. | Corpus leakage and reward hacking mimic reasoning improvement. | 2022 · 2024 · 2025 |
| Iterative, verifier, causal, and visual reasoning | Separates iterative decoding, verifiable-reward transfer, causal activation interventions, hierarchical rule agents, hidden-state answer selection, multimodal CoT, and rubric-decomposed visual feedback. | Verifier blind spots, answer leakage, judge dependence, extra sampling, or a fluent trace can improve benchmark scores without faithful or general reasoning. | 2026 · 2026 · 2026 · 2026 · 2026 · 2026 · 2026 |
| Anchor-based cross-tokenizer reasoning transfer | Uses anchors and residual-mass control instead of requiring identical vocabularies. | Math-only evidence and lossy first-subtoken mappings may not generalize to open generation. | 2026 |
| Reasoning budgets, executable outputs and scientific edits | Test-time scaling studies how extra reasoning changes perception and mathematics; EmbedCopilot checks whether plausible embedded-system answers actually execute; GRADE requires domain-correct and readable image edits. Together they motivate measuring compute, implementation success and constraint satisfaction rather than judging prose alone. | More tokens can reduce perceptual accuracy. Judge scores and physical execution use different protocols; GRADE strict accuracy is a conjunction of three maxima, not its mean reasoning score. | 2026 |
| Token-level direction and teacher consensus | BinaryOPD tests the sign of sampled-token teacher/student probability differences. C-MOPD resolves agreement and disagreement between task-specialist teachers rather than blindly averaging their signals. | Tested math/code token objectives are not a theorem that KL magnitudes or continuous feature residuals can always be discarded. Multiple teacher queries remain a training cost. | 2026 |
| Distillation changes capabilities, not only average scores | Distilled Pretraining separates access to teacher data from the effect of soft targets. It contrasts context copying with multi-sample solution coverage and tests entropy-based token routing. A student can improve on pass@k yet become less responsive to supplied context. | Matched token counts do not match total teacher–student compute. Context-copying experiments and pass@k are separate endpoints; routing results are not a universal recipe or demonstrated pathology transfer. | 2026 |
| Route additional correction by teacher–student disagreement | DrOPD preserves reverse KL at every student-generated position and adds forward KL at the batch’s highest-JS positions. The candidate union retains tokens preferred by either model. Read it after on-policy KD to distinguish uncertainty, disagreement and teacher-supported modes the student under-covers. | Math-only preprint, two related Qwen pairs. Top-k approximate losses, eight-response mean accuracy and logged training-step cost are separate evidence contracts; no demonstrated WSI transfer. | 2026 |
Concept lineage
Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsSelf-Consistency Improves Chain of Thought Reasoning in Language Models
One elicited rationale becomes a distribution of paths whose answers can be aggregated.
Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsScaling LLM Test-Time Compute Optimally Can Be More Effective than Scaling Model Parameters
Explicit tree search broadens into the general study of allocating and selecting inference-time computation.
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language ModelsDeepSeek-R1 Incentivizes Reasoning in LLMs through Reinforcement Learning
Domain-curated mathematical training and GRPO provide a precursor to broader reinforcement-learned reasoning behaviour.
Failure modes to audit
- Scoring only final answers while claiming valid reasoning.
- Comparing methods at unequal inference budgets.
- Using the target model as its own uncalibrated judge.
- Leaking benchmark solutions through rationale data.
- Treating longer traces as deeper reasoning.
Questions for every paper
- What additional computation is spent?
- How are candidates proposed and selected?
- Is process validity independently checked?
- How is contamination excluded?
Complete maintained paper route
Section titled “Complete maintained paper route”21 canonical readings · 2022–2026· route reviewed through October 2, 2026.
This is the complete structured list for this chapter, ordered chronologically from earlier foundations toward current work. The explanatory ladder above highlights the shortest conceptual route; this list also includes important branches, benchmarks, and current endpoints. It is generated from the same deduplicated catalog used by the AI background registry and My Library, so a future catalog update appears here automatically.
Show all 21 readings in maintained order
STaR: Bootstrapping Reasoning With Reasoning
Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman
Official artifacts: Official code
Metadata checked
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Jason Wei et al.
Metadata checked
Tree of Thoughts: Deliberate Problem Solving with Large Language Models
Shunyu Yao et al.
Official artifacts: Official code
Metadata checked
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes
Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister
Official artifacts: Official code
Metadata checked
Self-Consistency Improves Chain of Thought Reasoning in Language Models
Xuezhi Wang et al.
Metadata checked
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo
Official artifacts: Official models and implementation
Metadata checked
Let’s Verify Step by Step
Hunter Lightman et al.
Official artifacts: PRM800K dataset
Metadata checked
DeepSeek-R1 Incentivizes Reasoning in LLMs through Reinforcement Learning
Daya Guo et al.
Metadata checked
Scaling LLM Test-Time Compute Optimally Can Be More Effective than Scaling Model Parameters
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar
Metadata checked
On-policy Distillation with Verifiable Reward
Wenze Lin, Jiale Zhao, Xitai Jiang, Songde Rao, Yining Li, Shenzhi Wang, Bingxiang He, and Gao Huang
arXiv v1 preprint submitted 25 August 2026; evaluation is limited to the reported verifiable reasoning tasks.
Official artifacts: Official implementation
Metadata checked
Evolutionary Guided Decoding: Iterative Value Refinement for LLMs
Zhenhua Liu, Lijun Li, Ruizhe Chen, Yuxian Jiang, Tong Zhu, Zhaochen Su, Wenliang Chen, and Jing Shao
Peer-reviewed ACL 2026 long paper and Outstanding Paper award recipient.
Official artifacts: Earlier author manuscript
Metadata checked
CoSToM: Causal-oriented Steering for Intrinsic Theory-of-Mind Alignment in Large Language Models
Mengfan Li, Xuanhua Shi, and Yang Deng
Peer-reviewed ACL 2026 long paper and Best Theme Paper award recipient.
Official artifacts: Author manuscript · Official code
Metadata checked
HSCodeComp: A Realistic and Expert-level Agent Benchmark for Hierarchical Rule Application
Tian Lan, Yiqian Yang, Qianghuai Jia, Li Zhu, Hui Jiang, Hang Zhu, Weihua Luo, and Longyue Wang
Peer-reviewed ACL 2026 long paper and Best Resource Paper award recipient; later label-audit concerns should accompany benchmark use.
Official artifacts: Official benchmark repository · Official dataset · Independent primary label-audit manuscript
Metadata checked
Your Reasoning Model is Secretly a Reward Model - Optimization-Free Verification from Experience
Zhenwen Liang, Ruosen Li, Yujun Zhou, Linfeng Song, Dian Yu, Xinya Du, Haitao Mi, and Dong Yu
Peer-reviewed ACL 2026 long paper.
Metadata checked
Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Do
Zhuoran Jin, Kejian Zhu, Hongbang Yuan, Yupu Hao, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao
Peer-reviewed ACL 2026 long paper.
Official artifacts: Author manuscript
Metadata checked
V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning
Shulin Tian, Minglun Li, Yuhao Dong, Hao Ding, Jiarui Yao, Haiwen Diao, Jingkang Yang, Hongyuan Zhu, and Ziwei Liu
arXiv preprint first submitted 26 August 2026; not peer reviewed as of the audit date.
Official artifacts: Official project · Official code and data
Metadata checked
ACTD: Anchor-Based Cross-Tokenizer Distillation with Residual Regularization
Huiyi Zhang, Zijian Li, Xiaocheng Feng, Weitao Ma, Xiaoliang Yang, Yichong Huang, and Bing Qin
The arXiv record reports acceptance to the EMNLP 2026 main conference. No final ACL Anthology proceedings record was available at the 2026-09-02 audit, so methods and results remain pinned to arXiv v1 submitted 30 August 2026.
Official artifacts: Official implementation
Metadata checked
On Test-Time Scaling for Vision-Language Models
Fawaz Sammani, Tzoulio Chamiti, and Nikos Deligiannis
ECCV 2026 Spotlight verified in the official Vision-Language Models & Visual Foundation Models session; manuscript #4968 checked on 2026-09-12. Publisher version of record was not independently verified.
Official artifacts: ECCV Spotlight program · Conference manuscript
Metadata checked
Do We Really Need KL Divergence for On-Policy Distillation of Large Language Models?
Wenze Lin, Jiyuan Long, Jiale Zhao, Shenzhi Wang, Xitai Jiang, Ce Luo, Rui Lan, Qianli Ma, Fukang Wen, Hui Wu, Liyuan Chen, Shuoling Liu, Jiangpeng Yan, and Gao Huang
Preprint checked 2026-10-02. Empirical results concern autoregressive math and code distillation; the title is not a general theorem eliminating KL from all distillation.
Official artifacts: Versioned full text · Author implementation
Metadata checked
Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scaling
Sachin Goyal, David Lopez-Paz, and Kartik Ahuja
Official artifacts: Proceedings PDF · arXiv
Metadata checked
Divergence-Routed On-Policy Distillation for Language Models
Tiantian Peng, Guannan Lv, Hongyang Li, Qiuhe Hong, Hang Cheng, Hongjian Dou, Xiangyu Wu, and Yuyang Liu
Review pins arXiv v1 (14 October 2026). The manuscript says under review at ICLR 2027; this is not acceptance.
Official artifacts: Version 1 paper and appendices
Metadata checked