Reasoning with Sampling: Cutting at Decision Points
Abstract
Frontier reasoning models are produced by post-training base language models with reinforcement learning. Recent work has challenged this by showing that sampling from a sharpened version of the base model's distribution, a so-called power distribution, elicits comparable reasoning without additional training, curated datasets, or verifiers. However, making this method practical requires efficiently sampling from the power distribution. A sampler needs to "mix" to the power distribution, which necessitates moving between modes of the target distribution; intuitively, e.g., trying different reasoning strategies. The samplers proposed in prior works repeatedly select a "cut" position in the current reasoning trace uniformly at random and resample the suffix from that position onward. However, reasoning traces typically contain a few consequential decisions (e.g., the choice of proof strategy or algorithm), and we observe that a uniformly chosen cut tends to rewrite local details rather than revisit decision points. We introduce an algorithm (Entropy-Cut Metropolis-Hastings) that uses the base model's next-token entropy as a proxy to identify key decision points and resample from those positions. We empirically verify that entropy jumps are a useful proxy for decision points and, in a stylized model of reasoning, prove that our method's mixing time scales with the number of decisions in a trace rather than with the number of tokens, which can be much larger. Across MATH500, HumanEval, GPQA Diamond, and AIME26, our method consistently improves over baselines and RL-trained models, including best-of-N at the same token budget.
Related
- Towards Intelligent Investment Risk Modeling: Probabilistic Reasoning and Adaptive Decision Systems Under Uncertainty
- On the Sampling of Certain Dirichlet and Prime Sums at Gram Points and their Viscosity
- Pinpointing Pivot Points: The DECIDE Framework for Data Analysis Decision Importances
- Optimal Multi-way Decision Trees for Stratified Sampling in Online Controlled Experiments
- Transformation of Adaptive Multistage Sampling for Solving Finite-Horizon Markov Decision Processes with Unknown Model
BibTeX
@misc{zhou2026reasoning,
title = {{Reasoning with Sampling: Cutting at Decision Points}},
author = {Felix Zhou and Anay Mehrotra and Quanquan C. Liu},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2605.30327},
eprint = {2605.30327},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.30327},
}
RIS
TY - UNPB
TI - Reasoning with Sampling: Cutting at Decision Points
AU - Felix Zhou
AU - Anay Mehrotra
AU - Quanquan C. Liu
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2605.30327
UR - https://arxiv.org/abs/2605.30327
AB - Frontier reasoning models are produced by post-training base language models with reinforcement learning. Recent work has challenged this by showing that sampling from a sharpened version of the base model's distribution, a so-called power distribution, elicits comparable reasoning without additional training, curated datasets, or verifiers. However, making this method practical requires efficiently sampling from the power distribution. A sampler needs to "mix" to the power distribution, which necessitates moving between modes of the target distribution; intuitively, e.g., trying different reasoning strategies. The samplers proposed in prior works repeatedly select a "cut" position in the current reasoning trace uniformly at random and resample the suffix from that position onward. However, reasoning traces typically contain a few consequential decisions (e.g., the choice of proof strategy or algorithm), and we observe that a uniformly chosen cut tends to rewrite local details rather than revisit decision points. We introduce an algorithm (Entropy-Cut Metropolis-Hastings) that uses the base model's next-token entropy as a proxy to identify key decision points and resample from those positions. We empirically verify that entropy jumps are a useful proxy for decision points and, in a stylized model of reasoning, prove that our method's mixing time scales with the number of decisions in a trace rather than with the number of tokens, which can be much larger. Across MATH500, HumanEval, GPQA Diamond, and AIME26, our method consistently improves over baselines and RL-trained models, including best-of-N at the same token budget.
ER -