Can AI Forecast the Extremes? Evaluating Probabilistic Weather Models on Tropical Cyclones
Abstract
Weather prediction is undergoing an important shift, with data-driven AI models outperforming leading numerical weather prediction systems on average at a fraction of the computational cost. Yet, whether this success extends to extreme events remains contested. Earlier evaluations have exposed important shortcomings, but have largely focused on deterministic models and metrics, leaving the capabilities of the latest probabilistic models less understood. We conduct an extensive benchmark of tropical cyclone forecasting, comparing GenCast, FourCastNet 3, and FGN with ECMWF's operational physics-based ensemble, IFS-ENS. We develop a suite of probabilistic metrics and diagnostic analyses to assess cyclone tracking, intensity biases, uncertainty, and the wind-pressure relationship that characterizes storm structure. We find that, despite remaining limitations in intensity and ensemble spread, leading AI models substantially outperform IFS-ENS in cyclone tracking: FGN achieves better probabilistic location scores for 90% of evaluated storms in 2025. Further analyses reveal model-specific failure modes and suggest a lingering influence of pretraining data, pointing to opportunities for targeted improvement. Nonetheless, these results suggest that AI models are capable of capturing physical relationships that remain predictive well beyond typical weather conditions, motivating further investigation into how these capabilities emerge and how to strengthen them.
Related
- Basin-Specific Changes and Intensity of Tropical Cyclones
- Enhanced Influence of the North Pacific Meridional Mode on the Western North Pacific Tropical Cyclones in a Warming Climate
- Robust Regional Predictability of North Atlantic Tropical Cyclones on Seasonal Timescale
- Rare Tropical Cyclones: Tropical Cyclone Hidaya, A High‐Impact Low‐Latitude Event
- Eastward displacement of the Pacific–Japan pattern weakens its influence on summertime tropical cyclones affecting China
BibTeX
@misc{chalumeau2026forecast,
title = {{Can AI Forecast the Extremes? Evaluating Probabilistic Weather Models on Tropical Cyclones}},
author = {Nathan Chalumeau and Xavier Renard and Remi Meynadier and Hugo Rakotoarimanga and Marcin Detyniecki and Emmanuel de Bezenac},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.22077},
eprint = {2610.22077},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.22077},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Can AI Forecast the Extremes? Evaluating Probabilistic Weather Models on Tropical Cyclones
AU - Nathan Chalumeau
AU - Xavier Renard
AU - Remi Meynadier
AU - Hugo Rakotoarimanga
AU - Marcin Detyniecki
AU - Emmanuel de Bezenac
PY - 2026
DA - 2026/10/15
PB - arXiv
DO - 10.48550/arxiv.2610.22077
UR - https://arxiv.org/abs/2610.22077
AB - Weather prediction is undergoing an important shift, with data-driven AI models outperforming leading numerical weather prediction systems on average at a fraction of the computational cost. Yet, whether this success extends to extreme events remains contested. Earlier evaluations have exposed important shortcomings, but have largely focused on deterministic models and metrics, leaving the capabilities of the latest probabilistic models less understood. We conduct an extensive benchmark of tropical cyclone forecasting, comparing GenCast, FourCastNet 3, and FGN with ECMWF's operational physics-based ensemble, IFS-ENS. We develop a suite of probabilistic metrics and diagnostic analyses to assess cyclone tracking, intensity biases, uncertainty, and the wind-pressure relationship that characterizes storm structure. We find that, despite remaining limitations in intensity and ensemble spread, leading AI models substantially outperform IFS-ENS in cyclone tracking: FGN achieves better probabilistic location scores for 90% of evaluated storms in 2025. Further analyses reveal model-specific failure modes and suggest a lingering influence of pretraining data, pointing to opportunities for targeted improvement. Nonetheless, these results suggest that AI models are capable of capturing physical relationships that remain predictive well beyond typical weather conditions, motivating further investigation into how these capabilities emerge and how to strengthen them.
ER -