SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learning
Abstract
Physics-informed neural networks (PINNs) often face ill-conditioned objectives that limit high-accuracy training. Dense quasi-Newton methods improve local conditioning but require expensive optimizer state, while Kronecker-factored methods such as SOAP scale to larger networks but rely on periodic basis updates. We introduce \method, which augments SOAP-style preconditioning with a scalar secant-energy correction adapted to Kronecker geometry and an adaptive basis update followed by variance-state downscaling. We characterize the directional secant matching induced by the scalar correction and give a bound on variance-state mismatch across basis changes. Across eight PDE benchmarks, \method attains the lowest final residual on six, including Burgers and Boussinesq, while SOAP-family baselines perform better on Gray-Scott and Ginzburg-Landau. On Boussinesq, \method reaches a residual of 10−5 in 4.1 hours with 9.2 GB peak VRAM, while Adam does not reach this target within 14 hours. Three-seed L2 and H1 errors on four representative PDEs support the link between lower residuals and improved solution accuracy. These results position \method as a scalable option for stiff, high-accuracy physics-informed training, rather than a uniform replacement for existing optimizers.
Related
- AKAPINN: Adaptive Kolmogorov-Arnold Physics-Informed Neural Networks for approximating solutions to quasilinear partial differential equations
- Lie Symmetry-Guided Adaptive Sampling for Physics-Informed Neural Networks
- GAC-PINN: Geometry-Adaptive and Constraint-Enhanced Physics-Informed Neural Networks
- Comparison of neural and spline representations for physics-informed learning
- No-Harm Physics-Informed Inverse Learning with Residual-Calibrated Uncertainty
BibTeX
@misc{wang2026ssesoap,
title = {{SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learning}},
author = {Guangyuan Wang and Mads Toftrup and Sebastian Loeschcke and Yixuan Wang and Anima Anandkumar},
howpublished = {arXiv},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2608.29448},
eprint = {2608.29448},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.29448},
note = {CC BY-SA 4.0},
}
RIS
TY - UNPB
TI - SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learning
AU - Guangyuan Wang
AU - Mads Toftrup
AU - Sebastian Loeschcke
AU - Yixuan Wang
AU - Anima Anandkumar
PY - 2026
DA - 2026/08/29
PB - arXiv
DO - 10.48550/arxiv.2608.29448
UR - https://arxiv.org/abs/2608.29448
AB - Physics-informed neural networks (PINNs) often face ill-conditioned objectives that limit high-accuracy training. Dense quasi-Newton methods improve local conditioning but require expensive optimizer state, while Kronecker-factored methods such as SOAP scale to larger networks but rely on periodic basis updates. We introduce \method, which augments SOAP-style preconditioning with a scalar secant-energy correction adapted to Kronecker geometry and an adaptive basis update followed by variance-state downscaling. We characterize the directional secant matching induced by the scalar correction and give a bound on variance-state mismatch across basis changes. Across eight PDE benchmarks, \method attains the lowest final residual on six, including Burgers and Boussinesq, while SOAP-family baselines perform better on Gray-Scott and Ginzburg-Landau. On Boussinesq, \method reaches a residual of 10^−5 in 4.1 hours with 9.2 GB peak VRAM, while Adam does not reach this target within 14 hours. Three-seed L^2 and H^1 errors on four representative PDEs support the link between lower residuals and improved solution accuracy. These results position \method as a scalable option for stiff, high-accuracy physics-informed training, rather than a uniform replacement for existing optimizers.
ER -