Into the danger zone: stable extrapolation in high-dimensional function and operator learning

Abstract

Out-of-distribution (OOD) generalization is a central challenge in scientific machine learning. We study regression problems in which the test distribution differs from the training distribution and ask: under what assumptions on the target function or operator is stable extrapolation possible, and how far beyond the training domain can one extrapolate? Existing theory controls the test error through additive penalties measuring the discrepancy between the training and test distributions. Such guarantees show robustness to small distribution shifts, but can be very pessimistic in comparison to OOD performance observed empirically. We identify classes of holomorphic functions and operators for which the OOD generalization error converges at algebraic rates even in the presence of large distribution shifts. This phenomenon stems from the increasing smoothness of higher-index coordinates, leading to what we term a `blessing of high dimensionality'. For learning with either polynomials, deep neural networks or deep neural operators, we derive explicit rates for arbitrary test measures supported on suitable domains and quantify how the admissible domain depends on the underlying regularity of the function or operator. Our extrapolation guarantees are independent of the test distribution, depending only on its support. We also present a series of numerical experiments across a range of functions and operators that support the main theoretical findings.

L^p-extrapolation via non-local decay estimates: theory and applications to the generalized Stokes operator with rough coefficients
Luca Haardt
arXiv23 Sep 2026PreprintCC BY 4.0Full textAnalysis & PDEsarXiv
Compact sets in quasi-Banach function spaces and multilinear extrapolation of compactness on L^p(·)(w)
Spyridon Kakaroumpas, Stefanos Lappas
arXiv18 Sep 2026PreprintCC BY 4.0Full textAnalysis & PDEsarXiv
Multilinear extrapolation on Banach function spaces
Mingming Cao
arXiv26 Oct 2026PreprintarXiv licenceFull textAnalysis & PDEsarXiv
Risk-Calibrated Balancing for High-Dimensional Causal Extrapolation
Fenglin Yang, Haoran Lei, Yan Chen +1
arXiv2 Oct 2026PreprintarXiv licenceFull textStatisticsarXiv
K-theoretic invariants of four three-dimensional operator systems
Walter D. van Suijlekom
arXiv30 Sep 2026PreprintCC BY 4.0Full textAlgebra & number theory Analysis & PDEsarXiv

Cite

BibTeX RIS

BibTeX

@misc{adcock2026into,
  title = {{Into the danger zone: stable extrapolation in high-dimensional function and operator learning}},
  author = {Ben Adcock and Simone Brugiapaglia and Xuemeng Wang},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2609.36709},
  eprint = {2609.36709},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.36709},
  note = {CC BY 4.0},
}

RIS

TY  - UNPB
TI  - Into the danger zone: stable extrapolation in high-dimensional function and operator learning
AU  - Ben Adcock
AU  - Simone Brugiapaglia
AU  - Xuemeng Wang
PY  - 2026
DA  - 2026/10/03
PB  - arXiv
DO  - 10.48550/arxiv.2609.36709
UR  - https://arxiv.org/abs/2609.36709
AB  - Out-of-distribution (OOD) generalization is a central challenge in scientific machine learning. We study regression problems in which the test distribution differs from the training distribution and ask: under what assumptions on the target function or operator is stable extrapolation possible, and how far beyond the training domain can one extrapolate? Existing theory controls the test error through additive penalties measuring the discrepancy between the training and test distributions. Such guarantees show robustness to small distribution shifts, but can be very pessimistic in comparison to OOD performance observed empirically. We identify classes of holomorphic functions and operators for which the OOD generalization error converges at algebraic rates even in the presence of large distribution shifts. This phenomenon stems from the increasing smoothness of higher-index coordinates, leading to what we term a `blessing of high dimensionality'. For learning with either polynomials, deep neural networks or deep neural operators, we derive explicit rates for arbitrary test measures supported on suitable domains and quantify how the admissible domain depends on the underlying regularity of the function or operator. Our extrapolation guarantees are independent of the test distribution, depending only on its support. We also present a series of numerical experiments across a range of functions and operators that support the main theoretical findings.
ER  -

Abstract

Related

Cite