2D Stability Selection: Design Jittering to Exclude Correlated Irrelevant Features
Abstract
Stability Selection resamples the observations but leaves the design untouched, so it cannot reject an irrelevant feature that is correlated with the relevant ones in a way that violates the irrepresentable condition. Such a feature (a ``confounder'') is selected in a substantial fraction of resamples, and the exchangeability behind the false-positive bound of Stability Selection fails. We propose 2D Stability Selection (2D-SS), which aggregates selection frequencies over half-samples and over a grid of injected design noise, and reduces to Stability Selection at zero noise. Because input noise acts, in expectation, as Tikhonov regularisation, a jittered Lasso fit is a noisy elastic net whose support recovery is governed by a noise-indexed irrepresentable condition. Jittering can therefore repair designs in which confounders violate the Lasso condition, although it can also hurt. We give the noise level at which a violated feature's statistic vanishes (for uncorrelated relevant features), exact-recovery guarantees for a single fit, conditions under which aggregation over noise levels recovers the support, and a complementary-pairs bound on the expected number of false positives, carried over from Stability Selection to the noise-aggregated procedure, for a fixed threshold or a size-matched selector. The procedure takes no measurement-error covariance as input, although the guarantees assume i.i.d.\ Gaussian measurement error. In simulations 2D-SS excludes confounders that Stability Selection includes, and its size-matched version stays below the nominal closed-form false-positive bound in all 240 simulated cells, whereas Stability Selection exceeds it in 21; it does not generally improve on Stability Selection in F1 and is lower at low noise. It is useful when stable but wrong selections are the concern.
Related
- The Noise Is the Signal: Correlated Sampling Error Is Rank-Informative for Proxy Metric Selection
- Well-Posedness and Negative-Sobolev Stability of a Correlated-Noise Zakai Equation
- Nonperturbative Chiral Anomaly Cancellation and Irrelevant Operators
- Semi-similar Prandtl Flows near a Stagnation Point: Dynamical Selection and Nonlinear Stability
- ARISE: An adaptive residual-informed stability ensemble for feature selection in small-sample biomedical omics
BibTeX
@misc{nouraie2026stability,
title = {{2D Stability Selection: Design Jittering to Exclude Correlated Irrelevant Features}},
author = {Mahdi Nouraie and Houying Zhu and Samuel Muller},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.02205},
eprint = {2605.02205},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.02205},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - 2D Stability Selection: Design Jittering to Exclude Correlated Irrelevant Features
AU - Mahdi Nouraie
AU - Houying Zhu
AU - Samuel Muller
PY - 2026
DA - 2026/10/16
PB - arXiv
DO - 10.48550/arxiv.2605.02205
UR - https://arxiv.org/abs/2605.02205
AB - Stability Selection resamples the observations but leaves the design untouched, so it cannot reject an irrelevant feature that is correlated with the relevant ones in a way that violates the irrepresentable condition. Such a feature (a ``confounder'') is selected in a substantial fraction of resamples, and the exchangeability behind the false-positive bound of Stability Selection fails. We propose 2D Stability Selection (2D-SS), which aggregates selection frequencies over half-samples and over a grid of injected design noise, and reduces to Stability Selection at zero noise. Because input noise acts, in expectation, as Tikhonov regularisation, a jittered Lasso fit is a noisy elastic net whose support recovery is governed by a noise-indexed irrepresentable condition. Jittering can therefore repair designs in which confounders violate the Lasso condition, although it can also hurt. We give the noise level at which a violated feature's statistic vanishes (for uncorrelated relevant features), exact-recovery guarantees for a single fit, conditions under which aggregation over noise levels recovers the support, and a complementary-pairs bound on the expected number of false positives, carried over from Stability Selection to the noise-aggregated procedure, for a fixed threshold or a size-matched selector. The procedure takes no measurement-error covariance as input, although the guarantees assume i.i.d.\ Gaussian measurement error. In simulations 2D-SS excludes confounders that Stability Selection includes, and its size-matched version stays below the nominal closed-form false-positive bound in all 240 simulated cells, whereas Stability Selection exceeds it in 21; it does not generally improve on Stability Selection in F_1 and is lower at low noise. It is useful when stable but wrong selections are the concern.
ER -