Physics vs. AI in weather prediction: evaluating GraphCast, AIFS, and FuXi against an observation-corrected WRF model for flash floods
- 1University of Luxembourg (Luxembourg, Luxembourg)
Abstract
Data-driven artificial intelligence (AI) weather models are increasingly positioned as alternatives or complements to physics-based numerical weather prediction (NWP) systems, yet systematic model evaluation studies that compare both paradigms under controlled experimental conditions remain limited. Here we present a structured model evaluation framework applying standardised verification metrics to assess a regional WRF configuration against three publicly available AI models – GraphCast, AIFS Single 1.0, and FuXi – over three high-impact flood events in Luxembourg and the Greater Region (2016, 2018, and 2021), comprising around 90 simulated days per configuration and 92 119 matched forecast–observation pairs at a four-country network of 259 to 280 stations per event, each screened to be independent of the observations assimilated in that event (Fig. 1). The WRF model employs a three-dimensional variational (3D-VAR) data assimilation scheme ingesting GNSS Zenith Total Delay and conventional observations on a 6-hourly Rapid Update Cycle. All systems draw their initial and boundary information from the same ERA5 reanalysis, although the effective initial states are not identical: WRF is cycled and, in the assimilated configuration, corrected by observations, and ERA5 lateral boundary forcing of a limited-area model differs structurally from a global rollout. Categorical precipitation scores evaluated at thresholds from 1 to 20 mm (6 h) ⁻¹ and continuous temperature metrics at surface stations serve as verification targets. Data assimilation raises WRF's categorical precipitation skill at the 1 mm threshold (Critical Success Index: 0.364 to 0.389, a difference not resolved by a block bootstrap over 24 h blocks) and brings the near-surface temperature bias close to zero, although this benefit does not extend to heavier rainfall: at 10 mm (6 h) ⁻¹ the assimilated run scores below the unassimilated one. The degradation appears in all three events and is not ordered by the density of the assimilated observing network, so we report it without attributing a cause. At the 1 mm threshold AIFS achieves the highest detection rate (POD 0.835) and GraphCast the highest net categorical skill (CSI 0.424, against 0.415 for AIFS and 0.389 for the assimilated WRF), and AIFS and GraphCast reduce temperature RMSE by 19.0 % and 17.6 % relative to assimilated WRF at the subset of stations that also report temperature. This ordering reverses as the rainfall threshold increases: at 20 mm (6 h) ⁻¹ WRF attains the highest skill (CSI 0.160), ahead of AIFS (0.141) and GraphCast (0.114), while FuXi produces no exceedances at all. A weather-radar comparison over 27 6 h periods shows the same reversal in space: AIFS holds the highest mean per-period Fractions Skill Score at 1 mm (6 h) ⁻¹ at every neighbourhood width, but at 20 mm (6 h) ⁻¹ both WRF configurations are ahead of all three AI models on that estimator, and a WRF configuration leads on the pooled aggregate as well. The per-event breakdown reveals no skill degradation for GraphCast on the 2021 event, which lies outside its training period – indeed its skill is highest there – demonstrating useful skill on this later event without identifying the cause of the event-to-event differences; the same test cannot be applied to AIFS, whose fine-tuning window encompasses all three events. These results provide a replicable model evaluation methodology for benchmarking emerging data-driven NWP systems against observation-corrected regional models.
Versions
Related
- Evaluating LLMs for Translating Legacy Fortran Atmospheric Physics to GPU-Accelerated JAX
- GraphCast Skill and Systematic Biases in Indian Summer Monsoon Forecasts: Evaluation Against ERA5 and IMERG
- Automatic Extraction of a Generalized Physics‐Based Feature From Time‐Lapse Active Source Ultrasonic Waveforms for Prediction of Shear Stress in Laboratory Friction Experiments
- Data-efficient prediction of rare earth leaching from industrial residues with physics-constrained machine learning
- ClimTip-GML: A global bias-corrected and downscaled dataset for assessing impacts of climate tipping events
BibTeX
@article{rehman2026physics,
title = {{Physics vs. AI in weather prediction: evaluating GraphCast, AIFS, and FuXi against an observation-corrected WRF model for flash floods}},
author = {Haseeb ur Rehman and Félicia Norma Rebecca Teferle},
journal = {Geoscientific Model Development},
year = {2026},
month = oct,
doi = {10.5194/gmd-19-9831-2026},
url = {https://gmd.copernicus.org/articles/19/9831/2026/},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Physics vs. AI in weather prediction: evaluating GraphCast, AIFS, and FuXi against an observation-corrected WRF model for flash floods
AU - Haseeb ur Rehman
AU - Félicia Norma Rebecca Teferle
PY - 2026
DA - 2026/10/13
JO - Geoscientific Model Development
DO - 10.5194/gmd-19-9831-2026
UR - https://gmd.copernicus.org/articles/19/9831/2026/
AB - Data-driven artificial intelligence (AI) weather models are increasingly positioned as alternatives or complements to physics-based numerical weather prediction (NWP) systems, yet systematic model evaluation studies that compare both paradigms under controlled experimental conditions remain limited. Here we present a structured model evaluation framework applying standardised verification metrics to assess a regional WRF configuration against three publicly available AI models – GraphCast, AIFS Single 1.0, and FuXi – over three high-impact flood events in Luxembourg and the Greater Region (2016, 2018, and 2021), comprising around 90 simulated days per configuration and 92 119 matched forecast–observation pairs at a four-country network of 259 to 280 stations per event, each screened to be independent of the observations assimilated in that event (Fig. 1). The WRF model employs a three-dimensional variational (3D-VAR) data assimilation scheme ingesting GNSS Zenith Total Delay and conventional observations on a 6-hourly Rapid Update Cycle. All systems draw their initial and boundary information from the same ERA5 reanalysis, although the effective initial states are not identical: WRF is cycled and, in the assimilated configuration, corrected by observations, and ERA5 lateral boundary forcing of a limited-area model differs structurally from a global rollout. Categorical precipitation scores evaluated at thresholds from 1 to 20 mm (6 h) ⁻¹ and continuous temperature metrics at surface stations serve as verification targets. Data assimilation raises WRF's categorical precipitation skill at the 1 mm threshold (Critical Success Index: 0.364 to 0.389, a difference not resolved by a block bootstrap over 24 h blocks) and brings the near-surface temperature bias close to zero, although this benefit does not extend to heavier rainfall: at 10 mm (6 h) ⁻¹ the assimilated run scores below the unassimilated one. The degradation appears in all three events and is not ordered by the density of the assimilated observing network, so we report it without attributing a cause. At the 1 mm threshold AIFS achieves the highest detection rate (POD 0.835) and GraphCast the highest net categorical skill (CSI 0.424, against 0.415 for AIFS and 0.389 for the assimilated WRF), and AIFS and GraphCast reduce temperature RMSE by 19.0 % and 17.6 % relative to assimilated WRF at the subset of stations that also report temperature. This ordering reverses as the rainfall threshold increases: at 20 mm (6 h) ⁻¹ WRF attains the highest skill (CSI 0.160), ahead of AIFS (0.141) and GraphCast (0.114), while FuXi produces no exceedances at all. A weather-radar comparison over 27 6 h periods shows the same reversal in space: AIFS holds the highest mean per-period Fractions Skill Score at 1 mm (6 h) ⁻¹ at every neighbourhood width, but at 20 mm (6 h) ⁻¹ both WRF configurations are ahead of all three AI models on that estimator, and a WRF configuration leads on the pooled aggregate as well. The per-event breakdown reveals no skill degradation for GraphCast on the 2021 event, which lies outside its training period – indeed its skill is highest there – demonstrating useful skill on this later event without identifying the cause of the event-to-event differences; the same test cannot be applied to AIFS, whose fine-tuning window encompasses all three events. These results provide a replicable model evaluation methodology for benchmarking emerging data-driven NWP systems against observation-corrected regional models.
SN - 1991-9603
ER -