PreprintCC BY 4.0arXiv
Sharp Stationary Gaussian Approximation for Constant-Stepsize SGD
Abstract
We prove a sharp Gaussian approximation for the invariant law of constant-stepsize SGD with bounded additive noise generated by an exogenous uniformly ergodic Markov chain. For a smooth, strongly convex objective with a Lipschitz Hessian and nondegenerate long-run noise covariance, the centered iterate normalized by the square root of the stepsize is O(√α)-close in 1-Wasserstein distance to its limiting Gaussian. The proof combines blockwise Gaussian comparison with long-run contraction. A four-state example gives a matching lower bound although the one-time noise marginal is symmetric and every nonzero-lag autocovariance vanishes. In this example, an adjacent third-order mixed moment produces the leading correction.
Related
- Hamiltonian stationary Lagrangian surfaces of constant Gaussian curvature in complex space forms
- Moment-Accurate Gaussian Mixtures for Constant-Step Stochastic Approximation
- The Sharp Constant in the Second-Order Hydrogen Uncertainty Principle: Spectral Characterization and Quantitative Approximation
- Strong Gaussian Approximation of Constant Learning Rate SGD: Bias and Trajectory Inference
- Hamiltonian stationary Lagrangian surfaces in complex space forms with constant-length gradient of the mean curvature
BibTeX
@misc{seo2026sharp,
title = {{Sharp Stationary Gaussian Approximation for Constant-Stepsize SGD}},
author = {Junghoon Seo},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2609.39144},
eprint = {2609.39144},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.39144},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Sharp Stationary Gaussian Approximation for Constant-Stepsize SGD
AU - Junghoon Seo
PY - 2026
DA - 2026/10/01
PB - arXiv
DO - 10.48550/arxiv.2609.39144
UR - https://arxiv.org/abs/2609.39144
AB - We prove a sharp Gaussian approximation for the invariant law of constant-stepsize SGD with bounded additive noise generated by an exogenous uniformly ergodic Markov chain. For a smooth, strongly convex objective with a Lipschitz Hessian and nondegenerate long-run noise covariance, the centered iterate normalized by the square root of the stepsize is O(√α)-close in 1-Wasserstein distance to its limiting Gaussian. The proof combines blockwise Gaussian comparison with long-run contraction. A four-state example gives a matching lower bound although the one-time noise marginal is symmetric and every nonzero-lag autocovariance vanishes. In this example, an adjacent third-order mixed moment produces the leading correction.
ER -