Attention Is All You Need (to Avoid Spurious Oscillations)
Abstract
Can attention move a shock across several cells in one update without breaking it? We develop a conservative, fixed grid finite-volume scheme in which a CFL-conditioned attention flux selects upstream information according to the transport required by the current time step. One-dimensional inviscid Burgers transport is used as the central mechanism test: the same learned flux remains reliable in the conventional small-step regime and, with a time step four times larger, preserves sharp shocks while using one stage per update. A standard fifth-order WENO scheme with third-order strong-stability-preserving Runge-Kutta time integration (WENO-5+SSP-RK3) is included alongside controlled Forward Euler comparisons to separate flux selection from time integration. The learned attention shifts upstream with the local transport reach and becomes more selective near shocks; inference-time interventions and retrained ablations show that transport-scale information and state-dependent selection contribute directly to performance. Directional two-dimensional scalar Burgers transport and the one-dimensional shallow-water system then test whether the conservation-scale-selection principle transfers beyond the original scalar setting. The results support attention as a learnable information stencil for conservative large-step shock transport, while identifying finite candidate reach and problem-dependent robustness as the present limits.
Related
- On Spurious Mesh-Scale Oscillations Associated with Resonance-Cone Regimes in Wave-Plasma Simulations
- Macroscopic freezing of a vector Gaussian free field conditioned to avoid a ball
- Warned alike, AI agents avoid the less-crowded road while people take it
- Spurious spin nutation modes arising from truncated memory effects
- Bilayer Lithium Niobate Acoustic Resonators for Spurious-Free Wideband Operation at 6 GHz
BibTeX
@misc{jeong2026attention,
title = {{Attention Is All You Need (to Avoid Spurious Oscillations)}},
author = {Jinyoung Jeong and Joseph B. Choi and Xinlun Cheng and H. S. Udaykumar and Sanghun Choi and Stephen S. Baek},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.13531},
eprint = {2609.13531},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.13531},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Attention Is All You Need (to Avoid Spurious Oscillations)
AU - Jinyoung Jeong
AU - Joseph B. Choi
AU - Xinlun Cheng
AU - H. S. Udaykumar
AU - Sanghun Choi
AU - Stephen S. Baek
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2609.13531
UR - https://arxiv.org/abs/2609.13531
AB - Can attention move a shock across several cells in one update without breaking it? We develop a conservative, fixed grid finite-volume scheme in which a CFL-conditioned attention flux selects upstream information according to the transport required by the current time step. One-dimensional inviscid Burgers transport is used as the central mechanism test: the same learned flux remains reliable in the conventional small-step regime and, with a time step four times larger, preserves sharp shocks while using one stage per update. A standard fifth-order WENO scheme with third-order strong-stability-preserving Runge-Kutta time integration (WENO-5+SSP-RK3) is included alongside controlled Forward Euler comparisons to separate flux selection from time integration. The learned attention shifts upstream with the local transport reach and becomes more selective near shocks; inference-time interventions and retrained ablations show that transport-scale information and state-dependent selection contribute directly to performance. Directional two-dimensional scalar Burgers transport and the one-dimensional shallow-water system then test whether the conservation-scale-selection principle transfers beyond the original scalar setting. The results support attention as a learnable information stencil for conservative large-step shock transport, while identifying finite candidate reach and problem-dependent robustness as the present limits.
ER -