Cheap and Powerful Tests for Supervised Subspaces: Per-Component Inference for PLS
Abstract
Partial Least Squares (PLS) regression extracts a few outcome-aligned directions in a high-dimensional X and is widely used across applied science, but inference on the resulting fit is either expensive, biased and discouraged, or absent. We reduce inference to held-out OLS refits of the supervised subspace, a primitive shared by PLS, supervised PCA, and linear probes, and supply two tests using held-out correlations: a Nadeau-Bengio corrected asymptotic t-test as a fast approximation, and a permutation test with comparable power, finite-sample valid under outcome-predictor independence and iid rows. Held-out predictions are unchanged under any orthogonal rebasing of the supervised span, so an interpretable basis such as varimax inherits the joint claim but not a per-axis p-value; per-component claims come from a fixed-sequence test on the PLS extraction order. We validate on synthetic geometries, two NIR chemometric datasets, and cross-lingual word-embedding regressions; the exact test also transfers to supervised PCA and a ridge probe. The proposed tests have more power than CV-permutation-Q^2, at a fraction of its cost. A pre-run check on n and the spectrum of X says when the approximation is safe. We release a Rust library with Python, R, and Julia bindings, plus a Python text pipeline.
Related
- TwinPoSI: Statistical Twins for Powerful Post-Selection Inference on Population-Based Regression Coefficients
- Compressed Active Subspaces for Scalable Bayesian Inference
- Semiparametrically efficient semi-supervised statistical inference
- Spectra: Exact Component Transport for Test-Time Prior Adaptation in Simulation-Based Inference
- Co-Supervised Tree Synthesis for Interpretable Subgroup Identification and Honest Treatment Effect Inference in Randomized Trials
BibTeX
@misc{lenartowicz2026cheap,
title = {{Cheap and Powerful Tests for Supervised Subspaces: Per-Component Inference for PLS}},
author = {Paweł Lenartowicz and Hubert Plisiecki},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.36307},
eprint = {2609.36307},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.36307},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Cheap and Powerful Tests for Supervised Subspaces: Per-Component Inference for PLS
AU - Paweł Lenartowicz
AU - Hubert Plisiecki
PY - 2026
DA - 2026/09/28
PB - arXiv
DO - 10.48550/arxiv.2609.36307
UR - https://arxiv.org/abs/2609.36307
AB - Partial Least Squares (PLS) regression extracts a few outcome-aligned directions in a high-dimensional X and is widely used across applied science, but inference on the resulting fit is either expensive, biased and discouraged, or absent. We reduce inference to held-out OLS refits of the supervised subspace, a primitive shared by PLS, supervised PCA, and linear probes, and supply two tests using held-out correlations: a Nadeau-Bengio corrected asymptotic t-test as a fast approximation, and a permutation test with comparable power, finite-sample valid under outcome-predictor independence and iid rows. Held-out predictions are unchanged under any orthogonal rebasing of the supervised span, so an interpretable basis such as varimax inherits the joint claim but not a per-axis p-value; per-component claims come from a fixed-sequence test on the PLS extraction order. We validate on synthetic geometries, two NIR chemometric datasets, and cross-lingual word-embedding regressions; the exact test also transfers to supervised PCA and a ridge probe. The proposed tests have more power than CV-permutation-Q^2, at a fraction of its cost. A pre-run check on n and the spectrum of X says when the approximation is safe. We release a Rust library with Python, R, and Julia bindings, plus a Python text pipeline.
ER -