Precise Convergence Speed of Clipped SGD
Abstract
We present a tightened convergence analysis of clipped gradient descent on (L0, L1)-smooth functions, with quantitative constants. Building on the ideas of Koloskova et al (2023), we refactor several case disjunctions to reveal the central role of a control of the bias derived from fundamental properties of ℓ2-projection, simplifying proofs. We also extend the domain of validity from η≤ 1 / (9 β) to η< 1 /β where β= L0 + c L1 for clipping constant c, which matches the more traditional analysis of smooth functions. We strengthen the convergence criterion from ( mint < T 𝔼[lVert ∇ f(xt) rVert2] ) to ( 1/T ∑t < T 𝔼[lVert ∇ f(xt) rVert2] ) with matching speed, and lower the final achievable loss from 𝒪(min(σ2/c, σ)) to the more precise 6 min(σ2 /c, 3 σ).
Related
- Last-Iterate Convergence of Clipped SGD with Linear Decay
- On the Last-Iterate Convergence of Clipped Gradient Methods
- Impact of Feature Scaling on the Convergence Speed of Linear SVM Solvers : An Empirical Benchmark
- The speed of convergence in the Cooper-Dutle dueling game
- Propagation of opinions on a network. Convergence, spreading speed and patterns in the Daley-Kendall model
BibTeX
@misc{robin2026precise,
title = {{Precise Convergence Speed of Clipped SGD}},
author = {David A. R. Robin},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.29458},
eprint = {2609.29458},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.29458},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Precise Convergence Speed of Clipped SGD
AU - David A. R. Robin
PY - 2026
DA - 2026/09/24
PB - arXiv
DO - 10.48550/arxiv.2609.29458
UR - https://arxiv.org/abs/2609.29458
AB - We present a tightened convergence analysis of clipped gradient descent on (L_0, L_1)-smooth functions, with quantitative constants. Building on the ideas of Koloskova et al (2023), we refactor several case disjunctions to reveal the central role of a control of the bias derived from fundamental properties of ℓ_2-projection, simplifying proofs. We also extend the domain of validity from η≤ 1 / (9 β) to η< 1 /β where β= L_0 + c L_1 for clipping constant c, which matches the more traditional analysis of smooth functions. We strengthen the convergence criterion from ( min_t < T 𝔼[lVert ∇ f(x_t) rVert_2] ) to ( 1/T ∑_t < T 𝔼[lVert ∇ f(x_t) rVert_2] ) with matching speed, and lower the final achievable loss from 𝒪(min(σ^2/c, σ)) to the more precise 6 min(σ^2 /c, 3 σ).
ER -