Diagonal Attenuation: A Finite-Sample Correction for PCA
Abstract
Principal component analysis (PCA) can rotate away from its population target when a covariance matrix is estimated from limited data. We introduce diagonal attenuation, which preserves sample cross-covariances while reducing coordinatewise sample variances. The method is revealed exactly by averaging a linear full-output reconstruction loss over random input masks; studying the correction directly extends it beyond the range attainable by masking. We isolate the part of the random coupling between retained and omitted population directions that is contributed by sample-variance errors, and show how attenuation can reduce the resulting rotation. Under balanced marginal variances, we derive an explicit expected-risk theorem, uniform over the attenuation path for all sufficiently large finite samples, and obtain the asymptotically risk-minimizing strength. For general covariances, we characterize when attenuation leaves the population PCA subspace unchanged and give a risk theorem that also accounts for changing eigengaps and the population cost when the target moves. Simulations track this tradeoff from exact preservation back to PCA. Across local image patches, speech spectra, and smartphone acceleration, both mask-derived and direct attenuation improve PCA under two fitting-sample budgets, and one of them has the largest mean gain among seven methods in every data--budget cell. The full path selects strengths beyond the mask-derived boundary on 63%--95% of the subsamples.
Related
- Finite-Sample Rigidity for Left-Translated Similarity Orbits and Off-Diagonal Quadratic Realizations
- Reducibility and Diagonal Equality Strata for Finite Jacobi Pencils
- Finite Classification of Transversal Diagonal Gates Beyond Stabilizer Codes
- Finite-Dimensional Type I von Neumann Algebras in PyTorch: A GPU-Accelerated Framework for Random Block-Diagonal Operators
- Which Small-Sample Correction Should be Used When Analyzing Stepped-Wedge Designs With Time-Varying Treatment Effects?
BibTeX
@misc{sun2026diagonal,
title = {{Diagonal Attenuation: A Finite-Sample Correction for PCA}},
author = {Qiang Sun},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.05796},
eprint = {2609.05796},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.05796},
}
RIS
TY - UNPB
TI - Diagonal Attenuation: A Finite-Sample Correction for PCA
AU - Qiang Sun
PY - 2026
DA - 2026/09/05
PB - arXiv
DO - 10.48550/arxiv.2609.05796
UR - https://arxiv.org/abs/2609.05796
AB - Principal component analysis (PCA) can rotate away from its population target when a covariance matrix is estimated from limited data. We introduce diagonal attenuation, which preserves sample cross-covariances while reducing coordinatewise sample variances. The method is revealed exactly by averaging a linear full-output reconstruction loss over random input masks; studying the correction directly extends it beyond the range attainable by masking. We isolate the part of the random coupling between retained and omitted population directions that is contributed by sample-variance errors, and show how attenuation can reduce the resulting rotation. Under balanced marginal variances, we derive an explicit expected-risk theorem, uniform over the attenuation path for all sufficiently large finite samples, and obtain the asymptotically risk-minimizing strength. For general covariances, we characterize when attenuation leaves the population PCA subspace unchanged and give a risk theorem that also accounts for changing eigengaps and the population cost when the target moves. Simulations track this tradeoff from exact preservation back to PCA. Across local image patches, speech spectra, and smartphone acceleration, both mask-derived and direct attenuation improve PCA under two fitting-sample budgets, and one of them has the largest mean gain among seven methods in every data--budget cell. The full path selects strengths beyond the mask-derived boundary on 63%--95% of the subsamples.
ER -