Mean-Shift PCA by Knockoff Mean
Abstract
Removing noise is difficult, but adding noise is easy. In this work, we show how to eliminate mean-shift noisy components from PCA by deliberately introducing knockoff mean-shift perturbation. Standard PCA is highly sensitive to shifts in the sample mean: a small fraction of samples from a shifted distribution can cause large deviations in the leading principal components. In high-dimensional regimes, existing Robust PCA approaches cannot handle the mean-shift contamination structure inherent in the mixture model. Using tools from Random Matrix Theory, we prove that the mean-shift spikes are spectrally separable from the stable eigenvalues of the original covariance. Furthermore, the original eigenspace remains asymptotically invariant to the contamination, independent of the mixture weight. Exploiting this spectral stability, we propose a simple, two-stage PCA algorithm by adding knockoff mean that identifies and removes the mean-shift component using only standard PCA operations.
Related
- On a Class of Non-Local Elliptic-Transport Systems in Score-Guided Probability Flow on SE(3): Score-Singularity Resolution, Space-Time ADN Shift, and Markov Mean-Field Games
- Active Learning under Simulated Class Imbalance and Label Shift in Binary Semantic Segmentation
- On the sample complexity of the generalized hidden shift problem over arbitrary finite groups
- Numerical range of finite perturbations of a 2-periodic weighted shift
- Noncommutative commutant lifting, interpolation, and shift-invariant subspaces
BibTeX
@misc{li2026meanshift,
title = {{Mean-Shift PCA by Knockoff Mean}},
author = {Mengda Li and Zeng Li and Jianfeng Yao},
howpublished = {arXiv},
year = {2026},
month = may,
doi = {10.48550/arxiv.2605.25460},
eprint = {2605.25460},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.25460},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Mean-Shift PCA by Knockoff Mean
AU - Mengda Li
AU - Zeng Li
AU - Jianfeng Yao
PY - 2026
DA - 2026/05/25
PB - arXiv
DO - 10.48550/arxiv.2605.25460
UR - https://arxiv.org/abs/2605.25460
AB - Removing noise is difficult, but adding noise is easy. In this work, we show how to eliminate mean-shift noisy components from PCA by deliberately introducing knockoff mean-shift perturbation. Standard PCA is highly sensitive to shifts in the sample mean: a small fraction of samples from a shifted distribution can cause large deviations in the leading principal components. In high-dimensional regimes, existing Robust PCA approaches cannot handle the mean-shift contamination structure inherent in the mixture model. Using tools from Random Matrix Theory, we prove that the mean-shift spikes are spectrally separable from the stable eigenvalues of the original covariance. Furthermore, the original eigenspace remains asymptotically invariant to the contamination, independent of the mixture weight. Exploiting this spectral stability, we propose a simple, two-stage PCA algorithm by adding knockoff mean that identifies and removes the mean-shift component using only standard PCA operations.
ER -