PreprintCC BY 4.0arXiv
Interpreting relative utility for probabilistic predictions
Abstract
At a fixed threshold, relative utility (RU) measures the net-benefit gain of a prediction model over the better of treat-all and treat-none relative to the corresponding gain under perfect outcome classification. We illustrate that RU equal to 1 therefore represents perfect outcome classification at that fixed threshold, not perfect probabilistic prediction. However, even when every predicted probability equals the true probability, observed RU can equal 0. In a simple constant-risk setting, this occurs with probability approaching 1 as the sample size increases. Consequently, the distance from observed RU to 1 should not in general be interpreted as improvement achievable by a better prediction for binary probabilities.
Related
- Interpreting P values in clinical research: A practical guide for clinicians
- A Multiverse of Good and Bad Controls: Candidate Causal Graphs for Interpreting Model Robustness Analysis
- Interpreting Hierarchical Composite Endpoints with Survival and Longitudinal Outcomes: Application to Amyotrophic Lateral Sclerosis Trials
- A formula for the elasticity of substitution of a large class of smooth utility functions
- Interpreting Reasoning of Large Language Models via Partial Information Decomposition
BibTeX
@misc{hoessly2026interpreting,
title = {{Interpreting relative utility for probabilistic predictions}},
author = {Linard Hoessly},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.29133},
eprint = {2609.29133},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.29133},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Interpreting relative utility for probabilistic predictions
AU - Linard Hoessly
PY - 2026
DA - 2026/09/24
PB - arXiv
DO - 10.48550/arxiv.2609.29133
UR - https://arxiv.org/abs/2609.29133
AB - At a fixed threshold, relative utility (RU) measures the net-benefit gain of a prediction model over the better of treat-all and treat-none relative to the corresponding gain under perfect outcome classification. We illustrate that RU equal to 1 therefore represents perfect outcome classification at that fixed threshold, not perfect probabilistic prediction. However, even when every predicted probability equals the true probability, observed RU can equal 0. In a simple constant-risk setting, this occurs with probability approaching 1 as the sample size increases. Consequently, the distance from observed RU to 1 should not in general be interpreted as improvement achievable by a better prediction for binary probabilities.
ER -