Ranked Right, Sized Wrong: Ensemble Spread Fails Silently at Streamflow Extremes
Abstract
Deep learning models, particularly Long Short-Term Memory networks, are increasingly used for operational streamflow forecasting, with ensemble spread providing a signal of predictive uncertainty. As climate change increases the frequency of hydroclimatic extremes, these models will increasingly operate under flow conditions that are rare in their historical records. We therefore evaluate whether ensemble spread remains informative under increasing flows, examining both its magnitude relative to error and its ability to rank errors. Prediction error is approximately 2× the corrected spread at ordinary flows and up to 4× at the highest flows, with the discrepancy increasing with flow (log-log slope $1.30$, 95% CI: $[1.20,1.41]$). Under-dispersion is strongest in observed extreme flows beyond the training record, reaching an 18× discrepancy. This is pervasive and not explainable by conditional mean bias. Area Under the Sparsification Error, a standard distribution-free metric of uncertainty ranking, describing how well a model knows when it is uncertain, indicates near-oracle ranking for ensemble spread ($0.071$), yet ordering by predicted discharge alone nearly matches it. Informative uncertainty ranking can coexist with severe under-dispersion. In deployment, error ranking cannot be verified, yet the spread is available, and it is relied on as the uncertainty signal under increasingly extreme conditions, while systematically understating risk, most severely for the extreme events that matter most.
Related
- XAI-driven analysis of influencing factors in streamflow-based flood forecasting via fine tuned MTV19Anet
- Baseflow dominates river flow during droughts across the global streamflow network
- Towards generalized hydrological forecasting using transformer models for 120 h streamflow prediction
- The PIRAGUA_indicators dataset: Streamflow indices of the Pyrenees rivers (1950–2019)
- Multivariate SWAT streamflow and surface water storage calibration enables Upper Mississippi River Basin wetland change scenarios
BibTeX
@misc{gallant2026ranked,
title = {{Ranked Right, Sized Wrong: Ensemble Spread Fails Silently at Streamflow Extremes}},
author = {Justin Gallant and Tyler Wilson},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.20947},
eprint = {2610.20947},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.20947},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Ranked Right, Sized Wrong: Ensemble Spread Fails Silently at Streamflow Extremes
AU - Justin Gallant
AU - Tyler Wilson
PY - 2026
DA - 2026/10/14
PB - arXiv
DO - 10.48550/arxiv.2610.20947
UR - https://arxiv.org/abs/2610.20947
AB - Deep learning models, particularly Long Short-Term Memory networks, are increasingly used for operational streamflow forecasting, with ensemble spread providing a signal of predictive uncertainty. As climate change increases the frequency of hydroclimatic extremes, these models will increasingly operate under flow conditions that are rare in their historical records. We therefore evaluate whether ensemble spread remains informative under increasing flows, examining both its magnitude relative to error and its ability to rank errors. Prediction error is approximately 2× the corrected spread at ordinary flows and up to 4× at the highest flows, with the discrepancy increasing with flow (log-log slope $1.30$, 95% CI: $[1.20,1.41]$). Under-dispersion is strongest in observed extreme flows beyond the training record, reaching an 18× discrepancy. This is pervasive and not explainable by conditional mean bias. Area Under the Sparsification Error, a standard distribution-free metric of uncertainty ranking, describing how well a model knows when it is uncertain, indicates near-oracle ranking for ensemble spread ($0.071$), yet ordering by predicted discharge alone nearly matches it. Informative uncertainty ranking can coexist with severe under-dispersion. In deployment, error ranking cannot be verified, yet the spread is available, and it is relied on as the uncertainty signal under increasingly extreme conditions, while systematically understating risk, most severely for the extreme events that matter most.
ER -