SuperPCA: subspace analysis and an efficient algorithm for high-dimensional PCA
Abstract
Principal component analysis (PCA) is a fundamental tool to reduce the dimensionality of the data in many applications. PCA finds a few signal directions that contain most of the variability of the data by computing the eigenvectors of the sample covariance matrix. In this work, we focus on the spiked covariance model, in which the data vectors are defined by a few orthogonal signals plus an isotropic Gaussian noise, and our goal is to estimate one or more of the leading signals. Our main theoretical finding is that the subspace spanned by several leading eigenvectors of the sample covariance matrix contains significant information about the desired signals long before the individual eigenvectors converge to the population principal components. To prove this, we derive a posteriori bounds for the angle between the subspace spanned by the desired population signals and the subspace obtained from the sample using perturbation theory for singular vectors. This leads to a new algorithm, SuperPCA (SUbsPace subsamplER PCA), which capitalizes on an approximate eigenspace of the sample covariance matrix to find the leading signals far more efficiently and accurately than classical PCA in the high-dimensional, multi-signal setting. SuperPCA exploits only a small number of subsampled coordinates of the data, which can lead to tremendous savings in data acquisition cost, especially when the signals are approximately sparse. For the same number of measurements, SuperPCA can offer a factor $10$ improvement in accuracy compared to the classical PCA method.
Related
- Square-Root Clustered Lasso for High-Dimensional Linear Regression with Finite-Sample Theory and Efficient Algorithm Implementation
- An Iterative Active Subspace Approach for Model Order Reduction of Parametric Systems with High-Dimensional Parameter Spaces
- Geometric Fluctuations of the sinΘ Distance in High-Dimensional Principal Subspace Estimation
- ITHUIM: An efficient algorithm for mining inter-transaction high utility itemsets
- Sparse descent subspace-Newton thresholding algorithm for sparse optimization problems
BibTeX
@misc{haas2026superpca,
title = {{SuperPCA: subspace analysis and an efficient algorithm for high-dimensional PCA}},
author = {Irina-Beatrice Haas and Maike Meier and Yuji Nakatsukasa and Taejun Park},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.26406},
eprint = {2609.26406},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.26406},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - SuperPCA: subspace analysis and an efficient algorithm for high-dimensional PCA
AU - Irina-Beatrice Haas
AU - Maike Meier
AU - Yuji Nakatsukasa
AU - Taejun Park
PY - 2026
DA - 2026/09/22
PB - arXiv
DO - 10.48550/arxiv.2609.26406
UR - https://arxiv.org/abs/2609.26406
AB - Principal component analysis (PCA) is a fundamental tool to reduce the dimensionality of the data in many applications. PCA finds a few signal directions that contain most of the variability of the data by computing the eigenvectors of the sample covariance matrix. In this work, we focus on the spiked covariance model, in which the data vectors are defined by a few orthogonal signals plus an isotropic Gaussian noise, and our goal is to estimate one or more of the leading signals. Our main theoretical finding is that the subspace spanned by several leading eigenvectors of the sample covariance matrix contains significant information about the desired signals long before the individual eigenvectors converge to the population principal components. To prove this, we derive a posteriori bounds for the angle between the subspace spanned by the desired population signals and the subspace obtained from the sample using perturbation theory for singular vectors. This leads to a new algorithm, SuperPCA (SUbsPace subsamplER PCA), which capitalizes on an approximate eigenspace of the sample covariance matrix to find the leading signals far more efficiently and accurately than classical PCA in the high-dimensional, multi-signal setting. SuperPCA exploits only a small number of subsampled coordinates of the data, which can lead to tremendous savings in data acquisition cost, especially when the signals are approximately sparse. For the same number of measurements, SuperPCA can offer a factor $10$ improvement in accuracy compared to the classical PCA method.
ER -