Towards Calibrated Probabilistic Forecasts for Events of Interest via Outcome-Conditional Recalibration
Abstract
Calibration is an essential requirement for probabilistic predictions to be useful for decision making. While state-of-the-art prediction methods often yield miscalibrated predictive distributions, several post-hoc recalibration schemes have been proposed to generate calibrated predictions. However, popular recalibration schemes can conceal miscalibration in specific regions of the outcome space. Since particular outcomes, such as extreme events, often matter most for decision making, probabilistic predictions should be calibrated when evaluation is restricted to these outcomes. Hence, in this paper, we introduce outcome-conditional recalibration, a post-hoc method to recalibrate probabilistic predictions on user-defined regions of the outcome space. The method is simple, easy to implement, and can be applied to arbitrary predictive distributions. It works by applying the quantile recalibration approach of Kuleshov et al. (2018) to forecast conditional distributions, before rescaling these conditional distributions so that forecast event probabilities match empirical occurrence frequencies. This produces valid and continuous predictive distributions that are calibrated within each region of interest. Across regression benchmarks, we demonstrate that existing recalibration schemes do not necessarily yield calibrated predictions when interest is on particular outcomes, and that our approach improves outcome-conditional calibration relative to existing conditional and unconditional recalibration methods, while retaining competitive calibration overall. In an application to day-ahead electricity price forecasting, the approach substantially improves calibration when predicting negative prices, at negligible cost to forecast accuracy.
Related
- Learning joint probabilistic weather forecasts from station observations alone
- Probabilistic Streamflow Forecasts for Mississippi River Navigation Risk and Parametric Insurance
- Privacy-Preserving Distributed Online Dispatch of Low-Carbon Microgrids with TCN–BiLSTM Probabilistic Photovoltaic Forecasts
- Tail reliability in probabilistic electricity price forecasts: comparing model combination and conformal prediction in the GB day-ahead market
- Six-hourly Probabilistic Precipitation Forecasts Created From Hourly Probabilities
BibTeX
@misc{wessel2026towards,
title = {{Towards Calibrated Probabilistic Forecasts for Events of Interest via Outcome-Conditional Recalibration}},
author = {Jakob Benjamin Wessel and Sam Allen},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.10076},
eprint = {2610.10076},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.10076},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Towards Calibrated Probabilistic Forecasts for Events of Interest via Outcome-Conditional Recalibration
AU - Jakob Benjamin Wessel
AU - Sam Allen
PY - 2026
DA - 2026/10/07
PB - arXiv
DO - 10.48550/arxiv.2610.10076
UR - https://arxiv.org/abs/2610.10076
AB - Calibration is an essential requirement for probabilistic predictions to be useful for decision making. While state-of-the-art prediction methods often yield miscalibrated predictive distributions, several post-hoc recalibration schemes have been proposed to generate calibrated predictions. However, popular recalibration schemes can conceal miscalibration in specific regions of the outcome space. Since particular outcomes, such as extreme events, often matter most for decision making, probabilistic predictions should be calibrated when evaluation is restricted to these outcomes. Hence, in this paper, we introduce outcome-conditional recalibration, a post-hoc method to recalibrate probabilistic predictions on user-defined regions of the outcome space. The method is simple, easy to implement, and can be applied to arbitrary predictive distributions. It works by applying the quantile recalibration approach of Kuleshov et al. (2018) to forecast conditional distributions, before rescaling these conditional distributions so that forecast event probabilities match empirical occurrence frequencies. This produces valid and continuous predictive distributions that are calibrated within each region of interest. Across regression benchmarks, we demonstrate that existing recalibration schemes do not necessarily yield calibrated predictions when interest is on particular outcomes, and that our approach improves outcome-conditional calibration relative to existing conditional and unconditional recalibration methods, while retaining competitive calibration overall. In an application to day-ahead electricity price forecasting, the approach substantially improves calibration when predicting negative prices, at negligible cost to forecast accuracy.
ER -