Ensembles of Random Partition-Based Models under Covariate Shift
Abstract
Distribution shifts, including covariate shifts, are ubiquitous in real-world applications of machine learning. Empirical evidence suggests that, across a variety of models with and without downstream adaptation, ensembles tend to be more robust to covariate shift than single models. We extend the theory of covariate shift severity from single models to ensembles, providing theoretical justification for the empirical trends. We introduce new discrepancy measures of covariate shift severity to control the risk of partition-based kernel estimators. We then show that with sufficient ensemble size, these models can achieve minimax-optimal risk convergence rates under covariate shift. Our results suggest that modelers can expect to need larger ensemble sizes for more severe covariate shifts.
Related
- Conformalized Quantile Regression and Minimax Limits of Fixed-Score Calibration under Known Covariate Shift
- Backward Conformal Prediction under Covariate Shift
- Latent-Space Importance Weighting under Covariate Shift
- Finite-Sample Optimal Conformal Prediction Sets under Covariate Shift
- Sharp training-conditional coverage for conformal prediction under covariate shift
BibTeX
@misc{clemenssewall2026ensembles,
title = {{Ensembles of Random Partition-Based Models under Covariate Shift}},
author = {Mary Versa Clemens-Sewall and Eliza O'Reilly},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.05195},
eprint = {2611.05195},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.05195},
}
RIS
TY - UNPB
TI - Ensembles of Random Partition-Based Models under Covariate Shift
AU - Mary Versa Clemens-Sewall
AU - Eliza O'Reilly
PY - 2026
DA - 2026/11/03
PB - arXiv
DO - 10.48550/arxiv.2611.05195
UR - https://arxiv.org/abs/2611.05195
AB - Distribution shifts, including covariate shifts, are ubiquitous in real-world applications of machine learning. Empirical evidence suggests that, across a variety of models with and without downstream adaptation, ensembles tend to be more robust to covariate shift than single models. We extend the theory of covariate shift severity from single models to ensembles, providing theoretical justification for the empirical trends. We introduce new discrepancy measures of covariate shift severity to control the risk of partition-based kernel estimators. We then show that with sufficient ensemble size, these models can achieve minimax-optimal risk convergence rates under covariate shift. Our results suggest that modelers can expect to need larger ensemble sizes for more severe covariate shifts.
ER -