Distribution of hitting times for dissipative random dynamical systems on R^d , with application to stochastic gradient descent
Abstract
Machine Learning and more specifically Deep Learning involves solving large scale nonconvex optimization problems. Several algorithms have been proposed in the literature, that seem to achieve satisfactory practical efficiency for difficult instances, the Stochastic Gradient Method being the most rudimentary, while still outperforming more recent algorithms at a number of learning tasks. A major open question about the current methods used in deep learning is to understand their convergence properties. Following a line of previous works about the long time behavior of gradienttype algorithms, we present a new approach for studying the asymptotic properties of a wide family of methods from an ergodic theoretical viewpoint. Our main results include a study of the expected time for a stochastic optimisation algorithm to reach a certain small neighborhood of a minimizer and show that this reaching time distributes exponentially around its average, which is given by the inverse of the stationary measure of the target. The assumptions on the Stochastic Gradient noise include the Gaussian and the Sub-Exponential assumptions.
Related
- Fourth-Order Energy-Consistent Average Vector Field Discrete Gradient Methods for Dissipative Hamiltonian Systems
- Dissipative solutions for fibred Wasserstein gradient flows of Cahn-Hilliard-type energies
- Dissipative weak solutions to the three-dimensional stochastic Euler equations
- Supersymmetry of dissipative Bose-Fermi systems with application to Jaynes-Cummings and Dicke models
- Renyi's Relative Entropy Lyapunov Functional for Convergence to Invariant Measure in Dissipative Stochastic Hamiltonian Systems
BibTeX
@misc{galatolo2026distribution,
title = {{Distribution of hitting times for dissipative random dynamical systems on R^d , with application to stochastic gradient descent}},
author = {Stefano Galatolo and Stéphane Chrétien},
howpublished = {HAL},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2609.30274},
eprint = {2609.30274},
archivePrefix = {arXiv},
url = {https://hal.science/hal-05738581v1},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Distribution of hitting times for dissipative random dynamical systems on R^d , with application to stochastic gradient descent
AU - Stefano Galatolo
AU - Stéphane Chrétien
PY - 2026
DA - 2026/08/30
PB - HAL
DO - 10.48550/arxiv.2609.30274
UR - https://hal.science/hal-05738581v1
AB - Machine Learning and more specifically Deep Learning involves solving large scale nonconvex optimization problems. Several algorithms have been proposed in the literature, that seem to achieve satisfactory practical efficiency for difficult instances, the Stochastic Gradient Method being the most rudimentary, while still outperforming more recent algorithms at a number of learning tasks. A major open question about the current methods used in deep learning is to understand their convergence properties. Following a line of previous works about the long time behavior of gradienttype algorithms, we present a new approach for studying the asymptotic properties of a wide family of methods from an ergodic theoretical viewpoint. Our main results include a study of the expected time for a stochastic optimisation algorithm to reach a certain small neighborhood of a minimizer and show that this reaching time distributes exponentially around its average, which is given by the inverse of the stationary measure of the target. The assumptions on the Stochastic Gradient noise include the Gaussian and the Sub-Exponential assumptions.
ER -