Scaling of Capability and Efficiency at Inference Time in Large Reasoning Models
Abstract
Capability and efficiency are two key dimensions of reasoning in large language models (LLMs). Capability refers to the ability to solve a given problem correctly, whereas efficiency refers to the ability to do so with limited resources. When LLMs use Chain-of-Thought (CoT) reasoning to solve problems of controlled hardness, both the number of problems solved correctly and the number of tokens required to reach a correct answer depend on problem hardness and model size. However, how these factors jointly shape capability and efficiency remains poorly understood. Here, we use hierarchical Bayesian models to evaluate the capability and efficiency of LLMs from the DeepSeek-R1-Distill model family across four classes of arithmetic and algorithmic reasoning problems. At a fixed model size, the probability of correctly solving an instance decays approximately exponentially with instance size, our proxy for problem hardness. The decay scale grows sublinearly with model size, indicating that larger models are more capable, but that capability gains diminish with scale. Output length grows as a power law with instance size, which serves as a proxy for difficulty. However, the parameters of this power law do not vary systematically with model size, suggesting that larger models do not become more efficient. Together, these findings reveal potential limitations of naive scaling as a strategy for developing more capable AI systems: capability improves with diminishing returns, while efficiency shows little to no improvement.
Related
- Scaling LLM Agents for Materials Design through Hierarchical Collective Reasoning
- Dual-layer large-numerical-aperture metalenses with rotational zoom capability
- Stereotactic radiosurgery plan optimization with a human-in-the-loop reasoning large language model agent
- Coherent detection in 3D FMCW flash LIDAR with large FOV: part II – heterodyne efficiency in the image plane: erratum
- Large Enhancement of Spin-Flip Scattering Efficiency at Y3Fe5O12/Pt Interfaces Due to Vertical Confinement
BibTeX
@misc{laber2026scaling,
title = {{Scaling of Capability and Efficiency at Inference Time in Large Reasoning Models}},
author = {Moritz Laber and Zohair Shafi and Germans Savcisens and Brennan Klein and Matteo Chinazzi and Samuel V. Scarpino and Albert-László Barabási and Alessandro Vespignani and Tina Eliassi-Rad},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.27166},
eprint = {2609.27166},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.27166},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Scaling of Capability and Efficiency at Inference Time in Large Reasoning Models
AU - Moritz Laber
AU - Zohair Shafi
AU - Germans Savcisens
AU - Brennan Klein
AU - Matteo Chinazzi
AU - Samuel V. Scarpino
AU - Albert-László Barabási
AU - Alessandro Vespignani
AU - Tina Eliassi-Rad
PY - 2026
DA - 2026/09/24
PB - arXiv
DO - 10.48550/arxiv.2609.27166
UR - https://arxiv.org/abs/2609.27166
AB - Capability and efficiency are two key dimensions of reasoning in large language models (LLMs). Capability refers to the ability to solve a given problem correctly, whereas efficiency refers to the ability to do so with limited resources. When LLMs use Chain-of-Thought (CoT) reasoning to solve problems of controlled hardness, both the number of problems solved correctly and the number of tokens required to reach a correct answer depend on problem hardness and model size. However, how these factors jointly shape capability and efficiency remains poorly understood. Here, we use hierarchical Bayesian models to evaluate the capability and efficiency of LLMs from the DeepSeek-R1-Distill model family across four classes of arithmetic and algorithmic reasoning problems. At a fixed model size, the probability of correctly solving an instance decays approximately exponentially with instance size, our proxy for problem hardness. The decay scale grows sublinearly with model size, indicating that larger models are more capable, but that capability gains diminish with scale. Output length grows as a power law with instance size, which serves as a proxy for difficulty. However, the parameters of this power law do not vary systematically with model size, suggesting that larger models do not become more efficient. Together, these findings reveal potential limitations of naive scaling as a strategy for developing more capable AI systems: capability improves with diminishing returns, while efficiency shows little to no improvement.
ER -