Specification-Aware Predictive Uncertainty Decomposition

Abstract

Classical decompositions of predictive uncertainty, such as the information-theoretic one, are defined for a fixed representation x of the input object. They do not consider that x may be incomplete, nor do they measure how much the missing information can affect the prediction. In Large Language Models, users can clarify an underspecified input without any retraining. The clarified input can describe the user's unobserved intent z^* more accurately and change the predictive uncertainty about the outcome. In this paper, we formalize this setting probabilistically and derive a three-term decomposition of prediction risk into specification, residual aleatoric, and epistemic uncertainties. We show that at the original information level, when only x is available, the aleatoric uncertainty, represented by a Bayes risk, has two components. The first, which we call specification uncertainty, is the expected reduction in this Bayes risk from revealing the intent. The second is the residual aleatoric uncertainty, i.e., the Bayes risk that remains once the intent is known, averaged over its possible values z given x. The distinction is important, as specification uncertainty informs the decision to request clarification before making a prediction. We build approximations of these components and connect them to existing uncertainty measures. Our experiments across different domains and modalities show distinct roles for the three components. Specification uncertainty is the most informative for detecting underspecified inputs, while residual aleatoric uncertainty is the most informative for detecting in-distribution prediction errors.

Subjective Risk Decomposition: A New View for Uncertainty Quantification
Raghad Alamri, Michele Caprio, Gavin Brown
arXiv16 Sep 2026PreprintCC BY 4.0Full textStatisticsarXiv
Bagged Martingale Posteriors: Calibrated Uncertainty Quantification for Predictive Resampling
Hui Wang, Edwin Fong, David T. Frazier
arXiv24 Sep 2026PreprintarXiv licenceFull textStatisticsarXiv
Predictive Uncertainty for Neural CAE Surrogates
Kaustubh Tangsali, Mohammad Amin Nabian, Kelvin Lee +2
arXiv21 Sep 2026PreprintarXiv licenceFull textProbability Computational physicsarXiv
Nonnegativity Shapes Predictive Uncertainty: Complementary Model Fits and Quantile Construction
Jialin Li
arXiv1 Nov 2026PreprintarXiv licenceFull textStatisticsarXiv
A Hierarchy of Entropy-Shapley Games for Multivariate Predictive Uncertainty
Niklas Koenen, Claudia Battistin, Jeriek Van den Abeele +1
arXiv28 Sep 2026PreprintCC BY 4.0Full textStatisticsarXiv

Cite

BibTeX RIS

BibTeX

@misc{vashurin2026specificationaware,
  title = {{Specification-Aware Predictive Uncertainty Decomposition}},
  author = {Roman Vashurin and Mikhail Seleznyov and Assel Yermekova and Fedor Chernogorskii and Danila Zykov and Elena Tutubalina and Alexander Panchenko and Preslav Nakov and Nikita Kotelevskii and Maxim Panov},
  howpublished = {arXiv},
  year = {2026},
  month = nov,
  doi = {10.48550/arxiv.2611.02956},
  eprint = {2611.02956},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.02956},
  note = {CC BY 4.0},
}

RIS

TY  - UNPB
TI  - Specification-Aware Predictive Uncertainty Decomposition
AU  - Roman Vashurin
AU  - Mikhail Seleznyov
AU  - Assel Yermekova
AU  - Fedor Chernogorskii
AU  - Danila Zykov
AU  - Elena Tutubalina
AU  - Alexander Panchenko
AU  - Preslav Nakov
AU  - Nikita Kotelevskii
AU  - Maxim Panov
PY  - 2026
DA  - 2026/11/02
PB  - arXiv
DO  - 10.48550/arxiv.2611.02956
UR  - https://arxiv.org/abs/2611.02956
AB  - Classical decompositions of predictive uncertainty, such as the information-theoretic one, are defined for a fixed representation x of the input object. They do not consider that x may be incomplete, nor do they measure how much the missing information can affect the prediction. In Large Language Models, users can clarify an underspecified input without any retraining. The clarified input can describe the user's unobserved intent z^* more accurately and change the predictive uncertainty about the outcome. In this paper, we formalize this setting probabilistically and derive a three-term decomposition of prediction risk into specification, residual aleatoric, and epistemic uncertainties. We show that at the original information level, when only x is available, the aleatoric uncertainty, represented by a Bayes risk, has two components. The first, which we call specification uncertainty, is the expected reduction in this Bayes risk from revealing the intent. The second is the residual aleatoric uncertainty, i.e., the Bayes risk that remains once the intent is known, averaged over its possible values z given x. The distinction is important, as specification uncertainty informs the decision to request clarification before making a prediction. We build approximations of these components and connect them to existing uncertainty measures. Our experiments across different domains and modalities show distinct roles for the three components. Specification uncertainty is the most informative for detecting underspecified inputs, while residual aleatoric uncertainty is the most informative for detecting in-distribution prediction errors.
ER  -

Abstract

Related

Cite