Streaming PCA: averaging from a geometric perspective
Abstract
We study principal component analysis (PCA) under memory constraints, a setting that is increasingly important in large-scale data analysis. Our focus is on Oja's algorithm, which is a one-pass, memory-efficient algorithm requiring only O(p) storage in the rank-one case and O(pk) storage for k- PCA. The main goal is to develop a procedure based on Oja's algorithm that is optimal without prior knowledge of the eigengap, which is otherwise needed to tune the learning rates effectively. We do this by first introducing a geometric perspective on the convergence of k-PCA: by embedding the Oja iterates into the exterior space, we establish an exact equivalence between k-PCA in the ambient space and $1$-PCA in the exterior space. This geometric viewpoint enables us to establish several convergence guarantees, including one based only on a single learning-rate schedule. Building on this geometric perspective, we develop an averaging theory for k-PCA and show that the resulting averaged estimator is adaptive: it achieves the nearly optimal convergence rate without prior knowledge of the eigengap. As an application, we construct memory-efficient estimators for elliptical component analysis (ECA) \cite{han2014scale,han2018eca}. Simulation studies and real data analysis are conducted to demonstrate the benefits of our proposed algorithms.
Related
- Bayesian model averaging of risk set probabilities using a geometric representation of multivariate extremes
- Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errors
- A Unified Geometric Perspective on Zygmund's Maximal Function Conjecture and Related Questions for the Stein Hilbert Transform
- Streaming Hypergraph Coloring via Palette Sparsification
- Unique equivariant pseudo expectation and generalized Powers averaging for crossed product
BibTeX
@misc{pham2026streaming,
title = {{Streaming PCA: averaging from a geometric perspective}},
author = {Tuan Pham and Alessandro Rinaldo and Purnamrita Sarkar},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2609.23332},
eprint = {2609.23332},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.23332},
note = {CC0},
}
RIS
TY - UNPB
TI - Streaming PCA: averaging from a geometric perspective
AU - Tuan Pham
AU - Alessandro Rinaldo
AU - Purnamrita Sarkar
PY - 2026
DA - 2026/10/12
PB - arXiv
DO - 10.48550/arxiv.2609.23332
UR - https://arxiv.org/abs/2609.23332
AB - We study principal component analysis (PCA) under memory constraints, a setting that is increasingly important in large-scale data analysis. Our focus is on Oja's algorithm, which is a one-pass, memory-efficient algorithm requiring only O(p) storage in the rank-one case and O(pk) storage for k- PCA. The main goal is to develop a procedure based on Oja's algorithm that is optimal without prior knowledge of the eigengap, which is otherwise needed to tune the learning rates effectively. We do this by first introducing a geometric perspective on the convergence of k-PCA: by embedding the Oja iterates into the exterior space, we establish an exact equivalence between k-PCA in the ambient space and $1$-PCA in the exterior space. This geometric viewpoint enables us to establish several convergence guarantees, including one based only on a single learning-rate schedule. Building on this geometric perspective, we develop an averaging theory for k-PCA and show that the resulting averaged estimator is adaptive: it achieves the nearly optimal convergence rate without prior knowledge of the eigengap. As an application, we construct memory-efficient estimators for elliptical component analysis (ECA) \cite{han2014scale,han2018eca}. Simulation studies and real data analysis are conducted to demonstrate the benefits of our proposed algorithms.
ER -