Right, Wrong, or Unfinished: A Competing Risks Framework for Language-Model Reasoning under Token Budgets
Abstract
Assessing language-model performance under a token budget requires understanding both when a response will finish and whether it will be correct. We develop Competing Risks Analysis of Token Expenditure in Reasoning (CRATER), a statistically principled framework that treats correct and incorrect completion as distinct competing event types and responses unfinished at the generation limit as administratively right-censored. The fitted distribution provides probabilities of correct completion, incorrect completion, and remaining unfinished across token budgets, together with expected token consumption. We evaluate the proposed method using responses to MATH questions and demonstrate its value in two downstream applications. For routing before generation, CRATER guides the joint choice of a model configuration and token budget. Under hypothetical token prices, it reduces mean cost by 53.39 % relative to a strong reference, with observed correctcompletion rates of 96.42 % and 97.30 %, respectively. For repeated sampling, we combine CRATER's single-attempt success probabilities with an existing beta-binomial calibration approach to predict pass@k. Across both applications, CRATER delivers predictions at any specified token budget up to the generation limit from a common model of completion time and correctness, eliminating the need for separate parameter estimation for each budget.
Related
- Federated Language Models Under Bandwidth Budgets: Distillation Rates and Conformal Coverage
- Interpreting Reasoning of Large Language Models via Partial Information Decomposition
- A Distributional Framework for Behavioral Characterization and Risk Management of Large Language Models
- On the Multi-Index, Multi-Rate, and Multi-Phase Dynamics of Decoder-Only Language Models: A Unified Hybrid Framework for Generative and Agentic Systems
- A General Multimodal Probing Framework for Data Fusion and Prediction with Frozen Large Language Models: Evidence from Multimodal EHR Data
BibTeX
@misc{jiang2026right,
title = {{Right, Wrong, or Unfinished: A Competing Risks Framework for Language-Model Reasoning under Token Budgets}},
author = {Boen Jiang and Guanhua Fang and Wen Yu},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.05860},
eprint = {2611.05860},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.05860},
}
RIS
TY - UNPB
TI - Right, Wrong, or Unfinished: A Competing Risks Framework for Language-Model Reasoning under Token Budgets
AU - Boen Jiang
AU - Guanhua Fang
AU - Wen Yu
PY - 2026
DA - 2026/11/04
PB - arXiv
DO - 10.48550/arxiv.2611.05860
UR - https://arxiv.org/abs/2611.05860
AB - Assessing language-model performance under a token budget requires understanding both when a response will finish and whether it will be correct. We develop Competing Risks Analysis of Token Expenditure in Reasoning (CRATER), a statistically principled framework that treats correct and incorrect completion as distinct competing event types and responses unfinished at the generation limit as administratively right-censored. The fitted distribution provides probabilities of correct completion, incorrect completion, and remaining unfinished across token budgets, together with expected token consumption. We evaluate the proposed method using responses to MATH questions and demonstrate its value in two downstream applications. For routing before generation, CRATER guides the joint choice of a model configuration and token budget. Under hypothetical token prices, it reduces mean cost by 53.39 % relative to a strong reference, with observed correctcompletion rates of 96.42 % and 97.30 %, respectively. For repeated sampling, we combine CRATER's single-attempt success probabilities with an existing beta-binomial calibration approach to predict pass@k. Across both applications, CRATER delivers predictions at any specified token budget up to the generation limit from a common model of completion time and correctness, eliminating the need for separate parameter estimation for each budget.
ER -