Moment-Accurate Gaussian Mixtures for Constant-Step Stochastic Approximation
Abstract
Local Gaussian models of constant-step learning predict output variability and expected losses, but weak convergence alone does not justify these moment predictions. We establish moment-accurate Gaussian mixtures by matching stationary energy with local Ornstein--Uhlenbeck limits, ruling out quadratic tail mass invisible to weak convergence. For step size a, the second-order Wasserstein error is o(√a), uniformly over invariant laws, using each law's actual root weights. The assumptions combine confinement, descent, finitely many hyperbolic equilibria and root continuity with finite-variance innovations. The result yields observable covariances, expected objective gaps and first-order mean shifts, while allowing singular covariances, compatible saddles and weights without a limit. For additive noise given by a fixed invertible transform of independent standardized Student t3 coordinates, symmetry gives an order-sharp √a smooth-test bound. Numerical transport calculations demonstrate the value of root-specific covariances; controlled SGD studies assess observable predictions across step sizes, batch sizes and model geometries.
Related
- Sharp Stationary Gaussian Approximation for Constant-Stepsize SGD
- The Bias of Nonlinear Two-Time-scale Stochastic Approximation under Constant Step-Sizes
- Gaussian Approximation and Multiplier Bootstrap for Federated Linear Stochastic Approximation
- Strong Gaussian Approximation of Constant Learning Rate SGD: Bias and Trajectory Inference
- Analysis and Approximation of Stochastic Multiscale Subdiffusion Driven by Fractional Gaussian Noise
BibTeX
@misc{li2026momentaccurate,
title = {{Moment-Accurate Gaussian Mixtures for Constant-Step Stochastic Approximation}},
author = {Xiaoli Li and Wei Biao Wu},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.05595},
eprint = {2610.05595},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.05595},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Moment-Accurate Gaussian Mixtures for Constant-Step Stochastic Approximation
AU - Xiaoli Li
AU - Wei Biao Wu
PY - 2026
DA - 2026/10/04
PB - arXiv
DO - 10.48550/arxiv.2610.05595
UR - https://arxiv.org/abs/2610.05595
AB - Local Gaussian models of constant-step learning predict output variability and expected losses, but weak convergence alone does not justify these moment predictions. We establish moment-accurate Gaussian mixtures by matching stationary energy with local Ornstein--Uhlenbeck limits, ruling out quadratic tail mass invisible to weak convergence. For step size a, the second-order Wasserstein error is o(√a), uniformly over invariant laws, using each law's actual root weights. The assumptions combine confinement, descent, finitely many hyperbolic equilibria and root continuity with finite-variance innovations. The result yields observable covariances, expected objective gaps and first-order mean shifts, while allowing singular covariances, compatible saddles and weights without a limit. For additive noise given by a fixed invertible transform of independent standardized Student t_3 coordinates, symmetry gives an order-sharp √a smooth-test bound. Numerical transport calculations demonstrate the value of root-specific covariances; controlled SGD studies assess observable predictions across step sizes, batch sizes and model geometries.
ER -