Assumption-lean logistic regression with missing covariates
Abstract
Missing covariates are frequently encountered in supervised learning problems, and classical methods for estimation using such data use carefully chosen imputation schemes for missing data, or likelihood approximations that lead to nonconvex M-estimation problems. These methods and their relatives are suitable for scenarios in which the covariate distribution is known, and more broadly, have enjoyed tremendous success in linear models. But even in basic nonlinear problems such as logistic regression in moderate dimensions, such methods can experience drastic failure modes when the covariate distribution is unknown. Motivated by the need for reliable alternatives, we consider the problem of parameter estimation in logistic regression with missing covariates. Crucially, we operate in the assumption-lean setting where the covariate distribution is unknown (but bounded). We design a stochastic approximation method that is based on Z-estimation with a novel monotone operator, and establish that our algorithm is computationally efficient and achieves provable signal recovery at parametric rates under the hypothesis that covariates are missing completely at random. Our theory sharply characterizes the ℓ22 risk of the estimator in terms of the missingness profile, accommodating heterogeneous observation probabilities. Importantly, it shows that our method always outperforms the de facto ``complete-case'' estimator that ignores observations with any missing data. Even in the setting with homogeneous missingness (in which each covariate is observed independently with probability q), our bounds exhibit intricate and nonstandard dependence on q that can yield significant improvements over using only complete cases. We complement our upper bounds with new information-theoretic lower bounds that show that this intricate dependence on q is fundamental in a minimax sense.
Related
- Byzantine-Robust Sparse Regression with Missing Covariates
- Augmented transfer regression learning for completely missing covariates
- Doubly robust target inference for generalized linear regression with completely missing covariates
- Assumption-Lean Inference for Spectral Differential Network Analysis of High-Dimensional Time Series
- Computing assumption-lean bounds for per-protocol effects
BibTeX
@misc{choudhury2026assumptionlean,
title = {{Assumption-lean logistic regression with missing covariates}},
author = {Jyotishka Ray Choudhury and Kabir Aladin Verchand and Richard J. Samworth and Ashwin Pananjady},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.07292},
eprint = {2610.07292},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.07292},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Assumption-lean logistic regression with missing covariates
AU - Jyotishka Ray Choudhury
AU - Kabir Aladin Verchand
AU - Richard J. Samworth
AU - Ashwin Pananjady
PY - 2026
DA - 2026/10/05
PB - arXiv
DO - 10.48550/arxiv.2610.07292
UR - https://arxiv.org/abs/2610.07292
AB - Missing covariates are frequently encountered in supervised learning problems, and classical methods for estimation using such data use carefully chosen imputation schemes for missing data, or likelihood approximations that lead to nonconvex M-estimation problems. These methods and their relatives are suitable for scenarios in which the covariate distribution is known, and more broadly, have enjoyed tremendous success in linear models. But even in basic nonlinear problems such as logistic regression in moderate dimensions, such methods can experience drastic failure modes when the covariate distribution is unknown. Motivated by the need for reliable alternatives, we consider the problem of parameter estimation in logistic regression with missing covariates. Crucially, we operate in the assumption-lean setting where the covariate distribution is unknown (but bounded). We design a stochastic approximation method that is based on Z-estimation with a novel monotone operator, and establish that our algorithm is computationally efficient and achieves provable signal recovery at parametric rates under the hypothesis that covariates are missing completely at random. Our theory sharply characterizes the ℓ_2^2 risk of the estimator in terms of the missingness profile, accommodating heterogeneous observation probabilities. Importantly, it shows that our method always outperforms the de facto ``complete-case'' estimator that ignores observations with any missing data. Even in the setting with homogeneous missingness (in which each covariate is observed independently with probability q), our bounds exhibit intricate and nonstandard dependence on q that can yield significant improvements over using only complete cases. We complement our upper bounds with new information-theoretic lower bounds that show that this intricate dependence on q is fundamental in a minimax sense.
ER -