Online Supervised Dimension Reduction with Random Features: Diagnostics and Computational Trade-offs
Abstract
Accurate optimization of a supervised spectral objective need not produce an accurate population subspace or a better predictive representation. We investigate these distinctions for Online Kernel Supervised Principal Component Analysis (OKSPCA), which combines a centered cross-moment in finite random-feature coordinates with an Adam-style orthonormal basis update for an established objective. Fixed-map consistency, concentration and perturbation results describe the estimator and its exact subspace; same-target comparisons then assess the practical iterate separately. Across six predictive benchmarks, performance depends on the declared pipeline: replacing the tracker with the exact empirical target leaves the two regression deficits largely unchanged. Direct classification-rank models capture nearly all terminal objective energy on average, but a saved intermediate state exhibits substantial geometric deviation; a controlled sample-size study further separates empirical accuracy from population recovery. In distinct numerical-service workloads, exact on-request computation is faster in the tested classification settings, whereas Adam saves time relative to the tested full thin-SVD service for some dense wider-regression requests, alongside persistent geometric error. These diagnostics limit explanations based solely on terminal optimization accuracy and distinguish numerical cost from quality, rank coverage and freshness; they establish neither practical-tracker convergence nor predictive or deployment benefits from basis availability.
Related
- Atelier: Learning Local Self-Supervised Features for CryoEM Volumes via Hypernetworks
- Dimension reduction for characterizing sexual dimorphism in biomechanics of the temporomandibular joint
- Robust conditional dimension reduction for dissimilarity data
- Algebraic Dimension and Reduction of LVMB manifolds
- A capillary problem, its dimension reduction, and its phase-field approximation
BibTeX
@misc{yao2026online,
title = {{Online Supervised Dimension Reduction with Random Features: Diagnostics and Computational Trade-offs}},
author = {Zhenlin Yao and Wei Xiong},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.20454},
eprint = {2609.20454},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.20454},
}
RIS
TY - UNPB
TI - Online Supervised Dimension Reduction with Random Features: Diagnostics and Computational Trade-offs
AU - Zhenlin Yao
AU - Wei Xiong
PY - 2026
DA - 2026/09/17
PB - arXiv
DO - 10.48550/arxiv.2609.20454
UR - https://arxiv.org/abs/2609.20454
AB - Accurate optimization of a supervised spectral objective need not produce an accurate population subspace or a better predictive representation. We investigate these distinctions for Online Kernel Supervised Principal Component Analysis (OKSPCA), which combines a centered cross-moment in finite random-feature coordinates with an Adam-style orthonormal basis update for an established objective. Fixed-map consistency, concentration and perturbation results describe the estimator and its exact subspace; same-target comparisons then assess the practical iterate separately. Across six predictive benchmarks, performance depends on the declared pipeline: replacing the tracker with the exact empirical target leaves the two regression deficits largely unchanged. Direct classification-rank models capture nearly all terminal objective energy on average, but a saved intermediate state exhibits substantial geometric deviation; a controlled sample-size study further separates empirical accuracy from population recovery. In distinct numerical-service workloads, exact on-request computation is faster in the tested classification settings, whereas Adam saves time relative to the tested full thin-SVD service for some dense wider-regression requests, alongside persistent geometric error. These diagnostics limit explanations based solely on terminal optimization accuracy and distinguish numerical cost from quality, rank coverage and freshness; they establish neither practical-tracker convergence nor predictive or deployment benefits from basis availability.
ER -