Forecast‐Error Diagnostics in Neural Weather Models
- 1Faculty of Mathematics and Physics University of Ljubljana Ljubljana Slovenia
- 2Research Department, Earth System Assimilation Section ECMWF Bonn Germany
Abstract
Deep learning (DL) weather prediction models offer notable advantages over traditional physics‐based models, including auto‐differentiability and low computational cost, enabling detailed diagnostics of forecast errors. Using our convolutional encoder–decoder model, ConvCastNet, we relax selected subdomains of the forecast fields systematically towards “true” weather states (ECMWF ERA5 reanalysis) and monitor the forecast skill gain in other regions. Our results show that a medium‐range midlatitude forecast improves substantially when the stratosphere and boundary layer are relaxed, while relaxation of the tropical atmosphere has a limited effect. This underscores the need for a more accurate representation of the stratosphere and the planetary boundary layer to improve medium‐range weather predictability. Additionally, we investigate the relationship between the forecast‐error sensitivity to initial conditions and relaxation experiments. By utilising auto‐differentiability, we identify overlapping regions of large error sensitivity and strong forecast skill improvement from relaxation. Average midlatitude error sensitivity to initial conditions shows negligible influence from the Tropics, corroborating the results of the tropical relaxation experiments. The error sensitivity shows a physically consistent influence of upstream weather dynamics and sea‐surface temperatures on forecast accuracy. The latter also highlights the importance of representing the atmosphere–ocean coupling accurately in numerical weather prediction models. This combined approach could provide valuable heuristics for diagnosing neural model errors and guiding targeted model improvements.
Related
- AIFS Single 1.1.0: an update to ECMWF's machine-learned weather forecast model AIFS
- Forecast Skill Is Not Decision Skill: Evidence From Weather‐Dependent Decision Tasks
- Forecast lead time and the timing of crew deployment for weather-driven power outages using AI weather models
- Can AI Forecast the Extremes? Evaluating Probabilistic Weather Models on Tropical Cyclones
- How well can we forecast local magnetic ground perturbations with existing space weather monitoring resources?
BibTeX
@article{perkan2026forecasterror,
title = {{Forecast‐Error Diagnostics in Neural Weather Models}},
author = {Uroš Perkan and Gregor Skok and Žiga Zaplotnik},
journal = {Quarterly Journal of the Royal Meteorological Society},
year = {2026},
month = jun,
doi = {10.1002/qj.70212},
url = {https://doi.org/10.1002/qj.70212},
note = {CC BY},
}
RIS
TY - JOUR
TI - Forecast‐Error Diagnostics in Neural Weather Models
AU - Uroš Perkan
AU - Gregor Skok
AU - Žiga Zaplotnik
PY - 2026
DA - 2026/06/02
JO - Quarterly Journal of the Royal Meteorological Society
DO - 10.1002/qj.70212
UR - https://doi.org/10.1002/qj.70212
AB - Deep learning (DL) weather prediction models offer notable advantages over traditional physics‐based models, including auto‐differentiability and low computational cost, enabling detailed diagnostics of forecast errors. Using our convolutional encoder–decoder model, ConvCastNet, we relax selected subdomains of the forecast fields systematically towards “true” weather states (ECMWF ERA5 reanalysis) and monitor the forecast skill gain in other regions. Our results show that a medium‐range midlatitude forecast improves substantially when the stratosphere and boundary layer are relaxed, while relaxation of the tropical atmosphere has a limited effect. This underscores the need for a more accurate representation of the stratosphere and the planetary boundary layer to improve medium‐range weather predictability. Additionally, we investigate the relationship between the forecast‐error sensitivity to initial conditions and relaxation experiments. By utilising auto‐differentiability, we identify overlapping regions of large error sensitivity and strong forecast skill improvement from relaxation. Average midlatitude error sensitivity to initial conditions shows negligible influence from the Tropics, corroborating the results of the tropical relaxation experiments. The error sensitivity shows a physically consistent influence of upstream weather dynamics and sea‐surface temperatures on forecast accuracy. The latter also highlights the importance of representing the atmosphere–ocean coupling accurately in numerical weather prediction models. This combined approach could provide valuable heuristics for diagnosing neural model errors and guiding targeted model improvements.
SN - 0035-9009
SN - 1477-870X
ER -