In-context learning from self-generated trajectories for adaptive model reduction
Abstract
High-fidelity simulations of complex physical systems are often too expensive for repeated prediction, design, and control. Reduced-order models address this computational cost by compressing high-dimensional dynamics into low-dimensional representations that can be evaluated rapidly, but they often lose accuracy when online dynamics drift beyond the offline training data. Adaptive methods address this limitation by updating the reduced subspace online using external, out-of-span information, such as full-order corrections or sensor snapshots. We discovered that a complementary and previously unexploited in-span adaptation channel exists within the current reduced subspace. To exploit this channel, we continually update the reduced representation using the model's own predictions through an incremental singular-value decomposition with a forgetting factor. This produces a trajectory-informed spectral preconditioner in which the reduced subspace remains unchanged, while the basis is reweighted and realigned according to the directions visited by the evolving dynamics. This internal reorganization prepares the basis to absorb future out-of-span corrections more effectively, improving adaptation without requiring additional external information. We expose the mechanism through a three-dimensional spiral example and demonstrate its benefits on nonlinear partial differential equations, including viscous Burgers and Fisher--KPP dynamics. We also discuss how in-span learning can be interpreted as a dynamical-systems analogue of in-context learning. More broadly, in-span learning suggests a new principle for computational science, revealing that model-generated trajectories contain more usable information than previously recognized.
Related
- Transformers as Adaptive Kernel Learners across Tasks for In-Context Learning
- Adaptive Mean Estimation by In-Context Learning: A Gradient-Flow Analysis
- Particle Trajectories Beneath Fully Nonlinear Waves Generated by Horizontal Seabed Motion
- Dimensional reduction associated with unchanged direction trajectories in flat fiber Robertson Walker spacetimes
- An adaptive weighted extrapolation neural network for dispersive shock waves generated by square-root initial condition of the KdV equation
BibTeX
@misc{hedayat2026incontext,
title = {{In-context learning from self-generated trajectories for adaptive model reduction}},
author = {Amirpasha Hedayat and Laura Balzano and Karthik Duraisamy},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2607.02937},
eprint = {2607.02937},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.02937},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - In-context learning from self-generated trajectories for adaptive model reduction
AU - Amirpasha Hedayat
AU - Laura Balzano
AU - Karthik Duraisamy
PY - 2026
DA - 2026/09/20
PB - arXiv
DO - 10.48550/arxiv.2607.02937
UR - https://arxiv.org/abs/2607.02937
AB - High-fidelity simulations of complex physical systems are often too expensive for repeated prediction, design, and control. Reduced-order models address this computational cost by compressing high-dimensional dynamics into low-dimensional representations that can be evaluated rapidly, but they often lose accuracy when online dynamics drift beyond the offline training data. Adaptive methods address this limitation by updating the reduced subspace online using external, out-of-span information, such as full-order corrections or sensor snapshots. We discovered that a complementary and previously unexploited in-span adaptation channel exists within the current reduced subspace. To exploit this channel, we continually update the reduced representation using the model's own predictions through an incremental singular-value decomposition with a forgetting factor. This produces a trajectory-informed spectral preconditioner in which the reduced subspace remains unchanged, while the basis is reweighted and realigned according to the directions visited by the evolving dynamics. This internal reorganization prepares the basis to absorb future out-of-span corrections more effectively, improving adaptation without requiring additional external information. We expose the mechanism through a three-dimensional spiral example and demonstrate its benefits on nonlinear partial differential equations, including viscous Burgers and Fisher--KPP dynamics. We also discuss how in-span learning can be interpreted as a dynamical-systems analogue of in-context learning. More broadly, in-span learning suggests a new principle for computational science, revealing that model-generated trajectories contain more usable information than previously recognized.
ER -