Estimating ensemble-mean anomaly correlations and the ratio of predictable components using an unbiased estimate of the predictable signal variance
Abstract
The correlation (r) between ensemble mean forecasts and observations is a frequently-used metric of forecast quality in the weather and climate prediction communities. However, uncorrected estimates of r are biased towards zero at finite ensemble size due to the influence of unpredictable noise that contaminates the sample ensemble mean. This note summarises established ensemble-size adjustment methods and characterises the behaviour of r∞, a correlation estimator that is constructed from an unbiased and ensemble-size-independent estimate of the signal variance. The only requirement is that forecast members are exchangeable with one another such that they can be interpreted as independent draws from a case-specific predictive distribution. The estimator r∞ does not require forecasts and observations to be statistically exchangeable, only that the unpredictable component of each member is uncorrelated with the observed truth, and thus works for both reliable and unreliable forecasts. In some situations, r∞ may be undefined (i.e. the estimated signal variance is negative) or fall outside the admissible range −1 ≤ r∞ ≤ 1 when the predictable signal is small relative to the sampling uncertainty. These inadmissible values can be interpreted as evidence that the true predictable signal cannot be resolved with the available ensemble size and number of cases. The same unbiased signal variance can also be used to estimate the correlation between the predictable signal and individual members to yield an ensemble-size-invariant estimator of the ratio of predictable components (RPC). In contrast, variance-based RPC estimates, including forms that correct only the model predictable component, are biased low by a factor that depends only on the ensemble size and model signal-to-noise ratio.
Related
- Marine heat waves in vulnerable coral reef regions are skillfully predictable on multi-year timescales
- A more predictable Madden-Julian Oscillation index derived from Koopman spectral analysis
- S2S-JEPA: Predicting the Predictable at Subseasonal-to-Seasonal Timescales
- Is Earthquake Replenishment Sufficient for Unbiased b-Value Estimation?
- Estimating the Completeness Magnitude m c and the b ‐Values in a Snap
BibTeX
@misc{roberts2026estimating,
title = {{Estimating ensemble-mean anomaly correlations and the ratio of predictable components using an unbiased estimate of the predictable signal variance}},
author = {Christopher David Roberts},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.07878},
eprint = {2611.07878},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.07878},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Estimating ensemble-mean anomaly correlations and the ratio of predictable components using an unbiased estimate of the predictable signal variance
AU - Christopher David Roberts
PY - 2026
DA - 2026/11/05
PB - arXiv
DO - 10.48550/arxiv.2611.07878
UR - https://arxiv.org/abs/2611.07878
AB - The correlation (r) between ensemble mean forecasts and observations is a frequently-used metric of forecast quality in the weather and climate prediction communities. However, uncorrected estimates of r are biased towards zero at finite ensemble size due to the influence of unpredictable noise that contaminates the sample ensemble mean. This note summarises established ensemble-size adjustment methods and characterises the behaviour of r_∞, a correlation estimator that is constructed from an unbiased and ensemble-size-independent estimate of the signal variance. The only requirement is that forecast members are exchangeable with one another such that they can be interpreted as independent draws from a case-specific predictive distribution. The estimator r_∞ does not require forecasts and observations to be statistically exchangeable, only that the unpredictable component of each member is uncorrelated with the observed truth, and thus works for both reliable and unreliable forecasts. In some situations, r_∞ may be undefined (i.e. the estimated signal variance is negative) or fall outside the admissible range −1 ≤ r_∞ ≤ 1 when the predictable signal is small relative to the sampling uncertainty. These inadmissible values can be interpreted as evidence that the true predictable signal cannot be resolved with the available ensemble size and number of cases. The same unbiased signal variance can also be used to estimate the correlation between the predictable signal and individual members to yield an ensemble-size-invariant estimator of the ratio of predictable components (RPC). In contrast, variance-based RPC estimates, including forms that correct only the model predictable component, are biased low by a factor that depends only on the ensemble size and model signal-to-noise ratio.
ER -