Wet-Biased Data Pipelines Mislead the Training and Evaluation of Generative Precipitation Nowcasting
Abstract
Generative models for precipitation nowcasting are trained and evaluated predominantly on wet-biased datasets, which over-represent precipitation events to emphasize rare extremes. Selecting test cases based on target observations is known to favor forecasts that match the selected distribution rather than the intended deployment distribution. We show analytically how this target-dependent sampling also shifts the conditional distribution learned by generative models. When training and test sampling match, even an ideal model learns precisely the distribution favored by evaluation, allowing the evaluation bias to conceal the training bias. Using a fixed satellite-to-radar diffusion model, we demonstrate that target-dependent test sampling can reverse the calibration assessment. The model appears under-predictive on a wet-biased test set but over-predictive on a deployment-representative test set. This result is replicated outside the model's training domain after a seven-year gap. Utilizing classifier-free guidance as an example, we demonstrate how the contradictory calibration assessment can misdirect downstream model development.
Related
- GenONet: A Generative operator Network for High-Resolution Precipitation Nowcasting
- Precipitation nowcasting model based on multi-source data fusion and efficient additive attention
- Diffusion-Based Refinement for Kilometer-Scale Probabilistic Precipitation Nowcasting
- IRENE: A Convolutional GRU Ensemble Model for Radar Precipitation Nowcasting over Italy
- Pointwise is Pointless? A Multimodal Ablation Study for Precipitation Nowcasting with Graph Neural Networks
BibTeX
@misc{metzl2026wetbiased,
title = {{Wet-Biased Data Pipelines Mislead the Training and Evaluation of Generative Precipitation Nowcasting}},
author = {Christoph Metzl and Stefan Kapsreiter and Kianusch Vahid Yousefnia and Virginia Poli and Richard Müller and Frank Krüger and Tobias Bölle},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.26486},
eprint = {2610.26486},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.26486},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Wet-Biased Data Pipelines Mislead the Training and Evaluation of Generative Precipitation Nowcasting
AU - Christoph Metzl
AU - Stefan Kapsreiter
AU - Kianusch Vahid Yousefnia
AU - Virginia Poli
AU - Richard Müller
AU - Frank Krüger
AU - Tobias Bölle
PY - 2026
DA - 2026/10/19
PB - arXiv
DO - 10.48550/arxiv.2610.26486
UR - https://arxiv.org/abs/2610.26486
AB - Generative models for precipitation nowcasting are trained and evaluated predominantly on wet-biased datasets, which over-represent precipitation events to emphasize rare extremes. Selecting test cases based on target observations is known to favor forecasts that match the selected distribution rather than the intended deployment distribution. We show analytically how this target-dependent sampling also shifts the conditional distribution learned by generative models. When training and test sampling match, even an ideal model learns precisely the distribution favored by evaluation, allowing the evaluation bias to conceal the training bias. Using a fixed satellite-to-radar diffusion model, we demonstrate that target-dependent test sampling can reverse the calibration assessment. The model appears under-predictive on a wet-biased test set but over-predictive on a deployment-representative test set. This result is replicated outside the model's training domain after a seven-year gap. Utilizing classifier-free guidance as an example, we demonstrate how the contradictory calibration assessment can misdirect downstream model development.
ER -