Convergence rate of stochastic gradient descent under Łojasiewicz type condition and polynomially controlled noise
Abstract
Stochastic gradient methods are widely used for solving optimization problems in many scientific fields. Recently, their popularity has risen due to the increase in machine learning's prominence. Their convergence is often studied under the convexity assumption, which is too restrictive in modern applications. In this paper, we investigate the almost sure convergence rate and expectation under a Łojasiewicz-type gradient domination condition. This framework includes both bounded and decaying noise, as well as noise with controlled polynomial growth. The obtained results extend existing convergence by allowing a broader class of noise sequences and explicitly capturing the effect of their asymptotic behavior on the convergence rate and applicability.
Related
- Whole-Sequence Convergence of Variable-Smoothing Full-Splitting Methods via a Lifted Kurdyka-Łojasiewicz Framework
- Kurdyka-Łojasiewicz-based convergence analysis with approximate weakly convex subdifferentials: bundle, quadratic penalty, and moving balls approximation methods
- A global stochastic maximum principle for forward-backward stochastic control systems with quadratic convex generator and unbounded terminal condition
- A noncommunicative Kalman condition for null controllability of backward stochastic parabolic systems
- Tight Stochastic Condition-Number Dependence in Nonconvex-Strongly-Concave Minimax Optimization
BibTeX
@misc{wegrzyn2026convergence,
title = {{Convergence rate of stochastic gradient descent under Łojasiewicz type condition and polynomially controlled noise}},
author = {Paweł Węgrzyn},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.23386},
eprint = {2610.23386},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.23386},
}
RIS
TY - UNPB
TI - Convergence rate of stochastic gradient descent under Łojasiewicz type condition and polynomially controlled noise
AU - Paweł Węgrzyn
PY - 2026
DA - 2026/10/16
PB - arXiv
DO - 10.48550/arxiv.2610.23386
UR - https://arxiv.org/abs/2610.23386
AB - Stochastic gradient methods are widely used for solving optimization problems in many scientific fields. Recently, their popularity has risen due to the increase in machine learning's prominence. Their convergence is often studied under the convexity assumption, which is too restrictive in modern applications. In this paper, we investigate the almost sure convergence rate and expectation under a Łojasiewicz-type gradient domination condition. This framework includes both bounded and decaying noise, as well as noise with controlled polynomial growth. The obtained results extend existing convergence by allowing a broader class of noise sequences and explicitly capturing the effect of their asymptotic behavior on the convergence rate and applicability.
ER -