Spatial Aggregation of ROC and Precision-Recall Curves
Abstract
Receiver Operating Characteristic (ROC) and Precision-Recall (PR) curves are widely used to assess the discrimination ability of forecasts for binary events, such as threshold exceedances or warnings of extreme events. In weather forecasting, forecasts are provided as spatial fields, yielding location-wise ROC and PR curves that are often aggregated to facilitate comparison. However, the effect of the aggregation strategy on performance assessment remains poorly understood. We investigate how different aggregation strategies for ROC and PR curves affect the assessment of discrimination ability. In particular, we identify conditions under which aggregation strategies satisfy two desirable properties for fair comparison: preservation of dominance between forecasts and preservation of concavity or achievability of the curves. We obtain sufficient conditions and propose two strategies satisfying them. They are compared with existing strategies from the literature, and we analyze their properties and highlight potential pitfalls that may lead to misleading interpretations. Based on these findings, we provide practical guidelines for the interpretation of aggregated ROC and PR curves. The proposed framework is illustrated with AI-based global weather forecasts, showing how different aggregation strategies can yield different rankings of competing forecasts.
Related
- Density-dependent aggregation underlies spatial self-organization in coastal dunes.
- A recall-optimized LSTM–XGBoost ensemble model for river flood forecasting using multivariate time-series data—case study in Sri Lanka
- How to aggregate indicators? An overview of aggregation methods
- From aggregation to stabilization: the role of biomolecular corona formation on polyethylene nanoparticles in saline environments
- Quantifying aggregation bias in marsh carbon flux estimates caused by rhizosphere oxygen heterogeneity
BibTeX
@misc{pic2026spatial,
title = {{Spatial Aggregation of ROC and Precision-Recall Curves}},
author = {Romain Pic and Zhongwei Zhang and Sebastian Engelke and Johanna Ziegel},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.19517},
eprint = {2609.19517},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.19517},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Spatial Aggregation of ROC and Precision-Recall Curves
AU - Romain Pic
AU - Zhongwei Zhang
AU - Sebastian Engelke
AU - Johanna Ziegel
PY - 2026
DA - 2026/09/17
PB - arXiv
DO - 10.48550/arxiv.2609.19517
UR - https://arxiv.org/abs/2609.19517
AB - Receiver Operating Characteristic (ROC) and Precision-Recall (PR) curves are widely used to assess the discrimination ability of forecasts for binary events, such as threshold exceedances or warnings of extreme events. In weather forecasting, forecasts are provided as spatial fields, yielding location-wise ROC and PR curves that are often aggregated to facilitate comparison. However, the effect of the aggregation strategy on performance assessment remains poorly understood. We investigate how different aggregation strategies for ROC and PR curves affect the assessment of discrimination ability. In particular, we identify conditions under which aggregation strategies satisfy two desirable properties for fair comparison: preservation of dominance between forecasts and preservation of concavity or achievability of the curves. We obtain sufficient conditions and propose two strategies satisfying them. They are compared with existing strategies from the literature, and we analyze their properties and highlight potential pitfalls that may lead to misleading interpretations. Based on these findings, we provide practical guidelines for the interpretation of aggregated ROC and PR curves. The proposed framework is illustrated with AI-based global weather forecasts, showing how different aggregation strategies can yield different rankings of competing forecasts.
ER -