Online Covariance Estimation for Averaged Stochastic Gradient Descent on Riemannian Manifolds
Abstract
We study online inference for averaged Riemannian stochastic gradient descent (RSGD) when the unknown parameter lies on a manifold. Although the averaged RSGD iterate is asymptotically normal, practical inference requires estimating the unknown covariance operator in its limiting distribution. Existing online batch-means methods cannot be applied directly because a general Riemannian manifold lacks an addition operation, and the tangent space containing the Riemannian gradient varies across iterations. Through vector transport, we construct overlapping and non-overlapping covariance estimators that adaptively combine information from successive batches while accounting for the changing tangent spaces. The proposed estimators are online, require neither Riemannian Hessian computations nor a prespecified terminal sample size, and attain the same convergence rates as their Euclidean counterparts (Zhu et al., 2023). Together with asymptotic normality, these estimators enable online statistical inference for RSGD on manifolds. We illustrate the proposed framework through a detailed study of streaming principal component analysis on Grassmann manifolds.
Related
- Covariance and Principal Component Analysis on Riemannian Manifolds and Graphs
- Kernel Density Regression Estimation with Covariates on Riemannian Manifolds
- Smoothness estimation for Whittle–Matérn processes on closed Riemannian manifolds
- A Stochastic Riemannian Alternating Descent Ascent Method for Nonsmooth Composite Expectation Optimization on Riemannian Manifolds
- Riemannian Optimization on Unknown Manifolds via Stochastic Proximal Refinement
BibTeX
@misc{guan2026online,
title = {{Online Covariance Estimation for Averaged Stochastic Gradient Descent on Riemannian Manifolds}},
author = {Jingrong Guan and Liwei Jiang and Sen Na},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.33419},
eprint = {2610.33419},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.33419},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Online Covariance Estimation for Averaged Stochastic Gradient Descent on Riemannian Manifolds
AU - Jingrong Guan
AU - Liwei Jiang
AU - Sen Na
PY - 2026
DA - 2026/10/24
PB - arXiv
DO - 10.48550/arxiv.2610.33419
UR - https://arxiv.org/abs/2610.33419
AB - We study online inference for averaged Riemannian stochastic gradient descent (RSGD) when the unknown parameter lies on a manifold. Although the averaged RSGD iterate is asymptotically normal, practical inference requires estimating the unknown covariance operator in its limiting distribution. Existing online batch-means methods cannot be applied directly because a general Riemannian manifold lacks an addition operation, and the tangent space containing the Riemannian gradient varies across iterations. Through vector transport, we construct overlapping and non-overlapping covariance estimators that adaptively combine information from successive batches while accounting for the changing tangent spaces. The proposed estimators are online, require neither Riemannian Hessian computations nor a prespecified terminal sample size, and attain the same convergence rates as their Euclidean counterparts (Zhu et al., 2023). Together with asymptotic normality, these estimators enable online statistical inference for RSGD on manifolds. We illustrate the proposed framework through a detailed study of streaming principal component analysis on Grassmann manifolds.
ER -