Rare-Feature-Adaptive Subsampling for Big Data Regression
Abstract
Rare binary features cause instability and slow convergence in regression. Subsampling massive datasets may discard observations carrying rare-feature information. We show that rare-feature coefficient convergence rates depend on feature prevalence and the inclusion probabilities assigned to rare observations. This finding motivates subsampling designs that adapt sampling probabilities to feature rarity. We first study optimal rare-census designs, which extend the negative subsampling approach by retaining every rare observation. However, they may face practical constraints. We then propose a computationally efficient rarity-adaptive L-optimal design that incorporates inverse prevalence scaling. Because implementing optimal designs requires pilot estimates whose quality depends on adequate representation of rare features, we propose a balanced design, a one-step rarity-adaptive design based only on binary features. It can be used alone, as a pilot, or combined with designs targeting other objectives. For implementation in two stages, we develop a union-sample estimator that reuses pilot observations while accounting for sample overlap and establish its asymptotic normality. Simulations and applications show improved stability and estimation efficiency over existing methods. One application evaluates designs that jointly protect rare outcomes and rare features.
Related
- Central limit theorems for high-dimensional ridge regression under uniform subsampling
- Transfer Learning with Heterogeneous Feature Spaces in Linear Regression
- Benign Overfitting Property of the Minimum ℓq -norm Interpolant Estimator in Regression Problem via Feature Space Decomposition
- Efficient Benchmarking Is Just Feature Selection and Multiple Regression
- Feature Priming in Online Linear Regression: Sparse-Regret Lower Bounds and Tight Coordinatewise Rates
BibTeX
@misc{dong2026rarefeatureadaptive,
title = {{Rare-Feature-Adaptive Subsampling for Big Data Regression}},
author = {Qingkai Dong and Jun Yan and Qiang Zhang and HaiYing Wang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.29083},
eprint = {2610.29083},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.29083},
}
RIS
TY - UNPB
TI - Rare-Feature-Adaptive Subsampling for Big Data Regression
AU - Qingkai Dong
AU - Jun Yan
AU - Qiang Zhang
AU - HaiYing Wang
PY - 2026
DA - 2026/10/21
PB - arXiv
DO - 10.48550/arxiv.2610.29083
UR - https://arxiv.org/abs/2610.29083
AB - Rare binary features cause instability and slow convergence in regression. Subsampling massive datasets may discard observations carrying rare-feature information. We show that rare-feature coefficient convergence rates depend on feature prevalence and the inclusion probabilities assigned to rare observations. This finding motivates subsampling designs that adapt sampling probabilities to feature rarity. We first study optimal rare-census designs, which extend the negative subsampling approach by retaining every rare observation. However, they may face practical constraints. We then propose a computationally efficient rarity-adaptive L-optimal design that incorporates inverse prevalence scaling. Because implementing optimal designs requires pilot estimates whose quality depends on adequate representation of rare features, we propose a balanced design, a one-step rarity-adaptive design based only on binary features. It can be used alone, as a pilot, or combined with designs targeting other objectives. For implementation in two stages, we develop a union-sample estimator that reuses pilot observations while accounting for sample overlap and establish its asymptotic normality. Simulations and applications show improved stability and estimation efficiency over existing methods. One application evaluates designs that jointly protect rare outcomes and rare features.
ER -