How Does Distribution Shift Shape Pretraining Gains in Neural PDE Surrogates?
Abstract
Pretraining a neural PDE surrogate can reduce the amount of new CFD data needed when geometry or modeled physics changes. However, it remains unclear how different components of distribution shift affect this benefit. We pretrain a surrogate on 254,909 RANS solutions from one airfoil family and fine-tune it on a new family under two target settings with matched freestream ranges: the same Spalart-Allmaras (SA) modeling and SA with added eN transition modeling. At N=1000, the pretrained model matches the accuracy of a model trained from scratch on 3.25× as many samples for the same-SA target, but 2.58× as many for the transition-modeled target. By N=5000, this ordering reverses (1.56× versus 1.86×). At N=1000, sampling more distinct airfoils lowers error on both targets, but only for the same-SA target is the gain increase larger than the observed draw-to-draw variation (3.3× to 4.0×). These results show that pretraining value depends jointly on target-data budget, target-data coverage, and whether source and target differ in modeled physics.
Related
- Pretrain finite element method: A pretraining and warm-start framework for PDEs via physics-informed neural operators
- Neural Fourier Surrogates for Data Reuploading Quantum Neural Networks
- Polynomial neural surrogates for designing photonic quantum experiments
- A trust layer for fast pedestal-stability surrogates: calibrated error bars, out-of-distribution alarms and few-shot recalibration
- Benchmarking Quantum Convolutional Neural Networks and their classical surrogates on long-range strongly correlated models
BibTeX
@misc{bhargav2026does,
title = {{How Does Distribution Shift Shape Pretraining Gains in Neural PDE Surrogates?}},
author = {Pochinapeddi Sai Bhargav and Nithin Somasekharan and Rohit Sunil Kanchi and Sicheng He and Shaowu Pan},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.20814},
eprint = {2609.20814},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.20814},
}
RIS
TY - UNPB
TI - How Does Distribution Shift Shape Pretraining Gains in Neural PDE Surrogates?
AU - Pochinapeddi Sai Bhargav
AU - Nithin Somasekharan
AU - Rohit Sunil Kanchi
AU - Sicheng He
AU - Shaowu Pan
PY - 2026
DA - 2026/09/17
PB - arXiv
DO - 10.48550/arxiv.2609.20814
UR - https://arxiv.org/abs/2609.20814
AB - Pretraining a neural PDE surrogate can reduce the amount of new CFD data needed when geometry or modeled physics changes. However, it remains unclear how different components of distribution shift affect this benefit. We pretrain a surrogate on 254,909 RANS solutions from one airfoil family and fine-tune it on a new family under two target settings with matched freestream ranges: the same Spalart-Allmaras (SA) modeling and SA with added e^N transition modeling. At N=1000, the pretrained model matches the accuracy of a model trained from scratch on 3.25× as many samples for the same-SA target, but 2.58× as many for the transition-modeled target. By N=5000, this ordering reverses (1.56× versus 1.86×). At N=1000, sampling more distinct airfoils lowers error on both targets, but only for the same-SA target is the gain increase larger than the observed draw-to-draw variation (3.3× to 4.0×). These results show that pretraining value depends jointly on target-data budget, target-data coverage, and whether source and target differ in modeled physics.
ER -