Mean--Fluctuation Dynamics at the Edge of Stability
Abstract
We study the dynamics of gradient descent in the Edge of Stability regime, where the learning rate is large enough to induce persistent oscillations in the trajectory, which has been linked to better generalization performance. We introduce the mean--fluctuation dynamics, a tractable continuous-time model coupling the window-averaged trajectory to its fluctuation covariance. Among our contributions, we rigorously derive this model from gradient descent in a sharp-valley framework, characterize its stationary states and their linear stability, and establish precise connections with other effective dynamics. Numerical experiments illustrate these predictions and their finite-time limitations. We also study our model in the overparametrized regime of wide two-layer networks at a fixed learning rate, where we rigorously derive a kinetic equation describing weights and their fluctuations as a Wasserstein-2 gradient flow, for which we prove well-posedness, a mean-field limit, and conditional convergence results.
Related
- Local Algebraic Stability of Wasserstein Gradient Descent-Ascent and Mean Field Langevin Dynamics without Spectral Gap
- Monochromatic cycle partitions in 3-mean edge-colourings
- Linear Fluctuation of Interfaces in Glauber–Kawasaki Dynamics
- Mean curvature flows with cylindrical singularities II: stability and genericity
- A Stationary Mean-Field Model for Carbon Market Stability Reserve
BibTeX
@misc{courcel2026meanfluctuation,
title = {{Mean--Fluctuation Dynamics at the Edge of Stability}},
author = {Antonin Chodron de Courcel},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2606.05326},
eprint = {2606.05326},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.05326},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Mean--Fluctuation Dynamics at the Edge of Stability
AU - Antonin Chodron de Courcel
PY - 2026
DA - 2026/09/30
PB - arXiv
DO - 10.48550/arxiv.2606.05326
UR - https://arxiv.org/abs/2606.05326
AB - We study the dynamics of gradient descent in the Edge of Stability regime, where the learning rate is large enough to induce persistent oscillations in the trajectory, which has been linked to better generalization performance. We introduce the mean--fluctuation dynamics, a tractable continuous-time model coupling the window-averaged trajectory to its fluctuation covariance. Among our contributions, we rigorously derive this model from gradient descent in a sharp-valley framework, characterize its stationary states and their linear stability, and establish precise connections with other effective dynamics. Numerical experiments illustrate these predictions and their finite-time limitations. We also study our model in the overparametrized regime of wide two-layer networks at a fixed learning rate, where we rigorously derive a kinetic equation describing weights and their fluctuations as a Wasserstein-2 gradient flow, for which we prove well-posedness, a mean-field limit, and conditional convergence results.
ER -