Skip to content
DraconiticResearch
Search Sources API
194,216 studies31 sourcesUpdated just now
PreprintCC BY 4.0arXiv

A Rate Transfer Method: Convergence Rates in Stochastic Gradient Descent via Stochastic Calculus

  • Neil Walton
Open at sourcePDFDOICite

Abstract

We develop a rate-transfer method that converts finite-time Lyapunov estimates for an associated stochastic differential equation (SDE) into finite-time convergence bounds for stochastic gradient algorithms. Under generic moment and smoothness conditions, the convergence analysis of the stochastic gradient method reduces to a calculus argument verifying a Lyapunov generator inequality for the SDE. The method recovers established rates for classical stochastic gradient methods and provides a way to establish convergence rates for new optimization algorithms through calculus arguments applied to their associated SDEs.

Related

  1. Convergence rate of stochastic gradient descent under Łojasiewicz type condition and polynomially controlled noise
    Paweł Węgrzyn
    arXivPreprintarXiv licenceFull textNumerical analysis & scientific computingOptimisation & controlarXiv
  2. Large-Step Last-Iterate Convergence of Random-Reshuffling Stochastic Gradient Descent
    Zijun Chen, Zihan Zhang
    arXivPreprintarXiv licenceFull textOptimisation & controlStatisticsarXiv
  3. Convergence of stochastic gradient descent for learning linear neural networks
    Gabin Maxime Nguegnang, Bubacarr Bah, Holger Rauhut +1
    arXivPreprintarXiv licenceFull textOptimisation & controlarXiv
  4. Non-asymptotic Convergence of Stochastic Gradient Descent in Score-based Generative Models
    Stanislas Strasman, Sobihan Surendran, Sylvain Le Corff
    HALPreprintCC BY 4.0Full textStatisticsHALarXiv
  5. On the convergence of doubly stochastic block gradient descent method for linear inverse problems
    Yuxin Xia, Zehui Zhou
    arXivPreprintCC BY-NC-ND 4.0Full textNumerical analysis & scientific computingarXiv

Cite

BibTeXRIS
BibTeX
@misc{walton2026rate,
  title = {{A Rate Transfer Method: Convergence Rates in Stochastic Gradient Descent via Stochastic Calculus}},
  author = {Neil Walton},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2610.18739},
  eprint = {2610.18739},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.18739},
  note = {CC BY 4.0},
}
RIS
TY  - UNPB
TI  - A Rate Transfer Method: Convergence Rates in Stochastic Gradient Descent via Stochastic Calculus
AU  - Neil Walton
PY  - 2026
DA  - 2026/10/13
PB  - arXiv
DO  - 10.48550/arxiv.2610.18739
UR  - https://arxiv.org/abs/2610.18739
AB  - We develop a rate-transfer method that converts finite-time Lyapunov estimates for an associated stochastic differential equation (SDE) into finite-time convergence bounds for stochastic gradient algorithms. Under generic moment and smoothness conditions, the convergence analysis of the stochastic gradient method reduces to a calculus argument verifying a Lyapunov generator inequality for the SDE. The method recovers established rates for classical stochastic gradient methods and provides a way to establish convergence rates for new optimization algorithms through calculus arguments applied to their associated SDEs.
ER  -