Skip to content
DraconiticResearch
Search Sources API
201,818 studies31 sourcesUpdated 10 min ago
PreprintCC BYarXiv (Cornell University)

Can AI Weather Models Predict Beyond Two Weeks? A Quantitative Benchmark and Analysis of Long Rollouts

  • Fanny Lehmann1
  • Fırat Özdemir2
  • Yun Cheng2
  • Torsten Hoefler3
  • Sebastian Schemm4
  • Benedikt S. Soja5
  • Siddhartha Mishra6
6 affiliations
  1. 1ETH AI Center ETH Zurich
  2. 2Swiss Data Science Center ETH Zurich & EPFL
  3. 3Computing Lab ETH Zurich
  4. 4Dep. of Applied Mathematics and Theoretical Physics University of Cambridge
  5. 5Institute of Geodesy and Photogrammetry ETH Zurich
  6. 6Seminar for Applied Mathematics ETH Zurich
Open at sourcePDFDOICite

Abstract

While AI weather models excel at short-to-medium range forecasts (up to 15 days), they frequently suffer from ill-defined "instabilities" when rolled out over longer horizons. This work addresses the lack of a formal taxonomy by categorizing these failures into three distinct regimes: blow-up, drift, and loss of seasonality, through year-long rollouts of nine state-of-the-art AI weather models. Our analysis reveals that stability hinges on the treatment of small spatio-temporal scales: unstable models amplify high-frequency energy, while stable models act as denoisers when noise is added to their inputs. Far from reducing these models to mere stochastic parrots, our findings highlight that stable models generate unique weather trajectories, conditioned on the initial state. We verify our findings through ablation studies on architectural design choices, conducted using state-of-the-art Vision Transformer (ViT) AI weather model architectures.

Related

  1. A Global Event-Based Benchmark for Tide and Surge Models Beyond the Tide Gauge
    Amitay Sicherman, Patrick Ebel, Ivan D Haigh +8
    EarthArXivPreprintCC BY 4.0Full textOceanography & cryosphereEarthArXiv
  2. PhysMetrics.Weather: An Evaluation Framework for Physical Consistency in ML Weather Models
    Emma Kasteleyn, Timo Maier, Axel Lauer +3
    arXiv (Cornell University)PreprintLicence not statedFull text6 citedAtmospheric science & meteorologyOpenAlex
  3. Preserving Probabilistic Meaning in Weather-to-Text Generation
    Wanfeng Ding, Pengxin Lin, Hanif Bhuiyan
    EarthArXivPreprintPublisher termsFull textAtmospheric science & meteorologyEarthArXiv
  4. Skillful Seasonal Tropical Cyclone Prediction with AI Weather--Climate Models Forced by Dynamically Predicted SSTs
    Hiroyuki Murakami, Baoqiang Xiang
    EarthArXivPreprintPublisher termsFull textAtmospheric science & meteorologyEarthArXiv
  5. Rigorous uncertainty quantification of probabilistic AI weather forecasts with conformal prediction
    Anna Asch, Raphael Rossellini, Pedram Hassanzadeh +1
    arXiv (Cornell University)PreprintLicence not statedFull text6 citedAtmospheric science & meteorologyOpenAlex

Cite

BibTeXRIS
BibTeX
@misc{lehmann2026weather,
  title = {{Can AI Weather Models Predict Beyond Two Weeks? A Quantitative Benchmark and Analysis of Long Rollouts}},
  author = {Fanny Lehmann and Fırat Özdemir and Yun Cheng and Torsten Hoefler and Sebastian Schemm and Benedikt S. Soja and Siddhartha Mishra},
  howpublished = {arXiv (Cornell University)},
  year = {2026},
  month = may,
  doi = {10.48550/arxiv.2605.30184},
  url = {https://arxiv.org/abs/2605.30184},
  note = {CC BY},
}
RIS
TY  - UNPB
TI  - Can AI Weather Models Predict Beyond Two Weeks? A Quantitative Benchmark and Analysis of Long Rollouts
AU  - Fanny Lehmann
AU  - Fırat Özdemir
AU  - Yun Cheng
AU  - Torsten Hoefler
AU  - Sebastian Schemm
AU  - Benedikt S. Soja
AU  - Siddhartha Mishra
PY  - 2026
DA  - 2026/05/28
PB  - arXiv (Cornell University)
DO  - 10.48550/arxiv.2605.30184
UR  - https://arxiv.org/abs/2605.30184
AB  - While AI weather models excel at short-to-medium range forecasts (up to 15 days), they frequently suffer from ill-defined "instabilities" when rolled out over longer horizons. This work addresses the lack of a formal taxonomy by categorizing these failures into three distinct regimes: blow-up, drift, and loss of seasonality, through year-long rollouts of nine state-of-the-art AI weather models. Our analysis reveals that stability hinges on the treatment of small spatio-temporal scales: unstable models amplify high-frequency energy, while stable models act as denoisers when noise is added to their inputs. Far from reducing these models to mere stochastic parrots, our findings highlight that stable models generate unique weather trajectories, conditioned on the initial state. We verify our findings through ablation studies on architectural design choices, conducted using state-of-the-art Vision Transformer (ViT) AI weather model architectures.
SN  - 2331-8422
ER  -