The Multiple Timescales of Gradient Descent on the Edge of Stability: A Perturbative Derivation of the Central Flow
Abstract
The central flow of Cohen et al. (2025) is an empirically accurate continuous-time model of gradient descent at the edge of stability in deep learning, However, its derivation is heuristic. We propose a perturbative regime in which the central flow is the limit of gradient descent: we assume that the loss decomposes as f = g + εh; in the limit ε → 0, the dynamics of gradient descent with learning rate η converge to the gradient flow of h constrained to the minimizers of g of sharpness at most 2/η. Our approach is formal rather than rigorous; it treats gradient descent as a singularly perturbed dynamical system in ε. Three timescales emerge: a fast timescale of oscillations along the sharpest direction, an intermediate timescale of the self-stabilization mechanism, and a slow timescale of the dynamics along the minimizers of g-the central flow. Using the method of multiple scales, a classical formal method from singular perturbation theory, we derive the expansion of the dynamics in ε: the central flow emerges as the leading-order term in the expansion, while the self-stabilization mechanism appears in the next-order term. We study this mechanism beyond previous analyses: with a single eigenvalue at the edge of stability, we compute the slow drift of the energy of the fluctuations; with several eigenvalues at the edge of stability, we derive the self-stabilization system and explain why fluctuations persist.
Related
- Extended stability timescales for the 2D3C rotating stratified Boussinesq equations
- Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning
- On micro-macro derivation of traffic flow models with (A,B)-connection and its stability
- Micro-to-macro derivation and stability analysis of an Aw-Rascle-Zhang-type model with driver-dependent acceleration
- Sharp Gradient Stability for the Sobolev Trace Inequality
BibTeX
@misc{berthier2026multiple,
title = {{The Multiple Timescales of Gradient Descent on the Edge of Stability: A Perturbative Derivation of the Central Flow}},
author = {Raphaël Berthier},
howpublished = {HAL},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.01034},
eprint = {2609.01034},
archivePrefix = {arXiv},
url = {https://hal.science/hal-05733956v1},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - The Multiple Timescales of Gradient Descent on the Edge of Stability: A Perturbative Derivation of the Central Flow
AU - Raphaël Berthier
PY - 2026
DA - 2026/09/01
PB - HAL
DO - 10.48550/arxiv.2609.01034
UR - https://hal.science/hal-05733956v1
AB - The central flow of Cohen et al. (2025) is an empirically accurate continuous-time model of gradient descent at the edge of stability in deep learning, However, its derivation is heuristic. We propose a perturbative regime in which the central flow is the limit of gradient descent: we assume that the loss decomposes as f = g + εh; in the limit ε → 0, the dynamics of gradient descent with learning rate η converge to the gradient flow of h constrained to the minimizers of g of sharpness at most 2/η. Our approach is formal rather than rigorous; it treats gradient descent as a singularly perturbed dynamical system in ε. Three timescales emerge: a fast timescale of oscillations along the sharpest direction, an intermediate timescale of the self-stabilization mechanism, and a slow timescale of the dynamics along the minimizers of g-the central flow. Using the method of multiple scales, a classical formal method from singular perturbation theory, we derive the expansion of the dynamics in ε: the central flow emerges as the leading-order term in the expansion, while the self-stabilization mechanism appears in the next-order term. We study this mechanism beyond previous analyses: with a single eigenvalue at the edge of stability, we compute the slow drift of the energy of the fluctuations; with several eigenvalues at the edge of stability, we derive the self-stabilization system and explain why fluctuations persist.
ER -