BASIS: Batchwise Advantage Estimation from Single-Rollout Information Sharing for LLM Reasoning
Abstract
Reinforcement learning with verifiable rewards has become a standard recipe for improving the reasoning abilities of large language models. Existing algorithms face a tradeoff between computational efficiency and sample efficiency in value estimation and policy learning. We introduce BASIS, a critic-free post-training algorithm designed to address this tradeoff. At each online training step, BASIS samples only one rollout per prompt, but leverages rich information across prompts in the entire batch to improve value function estimation. Our experiments demonstrate that BASIS reduces MSE in value function estimation by 69% compared to REINFORCE++, a representative single-rollout baseline, and achieves lower MSE with one rollout than group mean estimators with 8 rollouts. This improvement in value estimation translates to better policy optimization: using substantially less training time, BASIS achieves performance close to multi-rollout GRPO-type baselines and often outperforms single-rollout REINFORCE-type baselines.
Related
- Interpreting Reasoning of Large Language Models via Partial Information Decomposition
- Information Lower Bound for Regime Boundary Estimation — Optimal Precision Limit at the n² Scale and Universal Constant c=1/8 in the Fixed-Breakpoint Model
- Estimation with Quantized Parameter Side-Information
- Mutual Information Estimation via Triangular Transport Maps: Finite Sample Guarantees and Sample Allocation
- Estimation of Quantum Umlaut Information
BibTeX
@misc{gong2026basis,
title = {{BASIS: Batchwise Advantage Estimation from Single-Rollout Information Sharing for LLM Reasoning}},
author = {Shijin Gong and Erhan Xu and Kai Ye and Giulia Livieri and Francesco Quinzan and Chengchun Shi},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2605.27293},
eprint = {2605.27293},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.27293},
}
RIS
TY - UNPB
TI - BASIS: Batchwise Advantage Estimation from Single-Rollout Information Sharing for LLM Reasoning
AU - Shijin Gong
AU - Erhan Xu
AU - Kai Ye
AU - Giulia Livieri
AU - Francesco Quinzan
AU - Chengchun Shi
PY - 2026
DA - 2026/09/15
PB - arXiv
DO - 10.48550/arxiv.2605.27293
UR - https://arxiv.org/abs/2605.27293
AB - Reinforcement learning with verifiable rewards has become a standard recipe for improving the reasoning abilities of large language models. Existing algorithms face a tradeoff between computational efficiency and sample efficiency in value estimation and policy learning. We introduce BASIS, a critic-free post-training algorithm designed to address this tradeoff. At each online training step, BASIS samples only one rollout per prompt, but leverages rich information across prompts in the entire batch to improve value function estimation. Our experiments demonstrate that BASIS reduces MSE in value function estimation by 69% compared to REINFORCE++, a representative single-rollout baseline, and achieves lower MSE with one rollout than group mean estimators with 8 rollouts. This improvement in value estimation translates to better policy optimization: using substantially less training time, BASIS achieves performance close to multi-rollout GRPO-type baselines and often outperforms single-rollout REINFORCE-type baselines.
ER -