Adaptive mixture variational inference for spike-and-slab regression
Abstract
Correlated predictors can support competing sparse explanations with similar predictions, making joint uncertainty about variable inclusion difficult to capture with mean-field approximations. We develop an adaptive fitting procedure for mixtures of product distributions in Gaussian regression with a point-mass spike-and-slab prior. It minimizes reverse Kullback-Leibler divergence directly on inclusion indicators and active coefficients, jointly refining component parameters and weights as the mixture grows. This avoids an additional divergence penalty on unused latent coefficients under independent augmentation. Our analysis relates approximation accuracy to mixture size, support coverage and dependence within supports, and establishes contraction, selection consistency and a Bernstein-von Mises approximation under explicit conditions on the prior, posterior concentration and variational error. On all 250 simulated datasets with exact posterior references, mixtures reduce errors in inclusion probabilities, grouped support probabilities and coefficient covariance relative to multistart mean field. Comparisons at fixed mixture size and common initialization favor direct joint refinement over augmented or restricted refinement in posterior divergence. Complete stagewise fitting can nevertheless be more accurate near collinearity. The results support direct joint refinement for posterior approximation while showing that local gains do not ensure superiority of the full adaptive search.
Related
- A Shape-Adaptive Gamma-Mixture Power Shanker Distribution: Boundary-Aware Inference, Censored Regression, and Predictive Validation
- Stochastic Variational Inference for Vine Copula Distributional Regression
- Adaptive Functional Clustering with Structured Dependence via Variational Inference
- Component-weighted sparse group LASSO for finite mixture regression models
- Variational Bayesian Tensor Decomposition With Discrete Mixture Prior for Unsourced Random Access
BibTeX
@misc{li2026adaptive,
title = {{Adaptive mixture variational inference for spike-and-slab regression}},
author = {Hanqing Li and Yaroslav Golub and Xuewen Lu},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.38656},
eprint = {2609.38656},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.38656},
}
RIS
TY - UNPB
TI - Adaptive mixture variational inference for spike-and-slab regression
AU - Hanqing Li
AU - Yaroslav Golub
AU - Xuewen Lu
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2609.38656
UR - https://arxiv.org/abs/2609.38656
AB - Correlated predictors can support competing sparse explanations with similar predictions, making joint uncertainty about variable inclusion difficult to capture with mean-field approximations. We develop an adaptive fitting procedure for mixtures of product distributions in Gaussian regression with a point-mass spike-and-slab prior. It minimizes reverse Kullback-Leibler divergence directly on inclusion indicators and active coefficients, jointly refining component parameters and weights as the mixture grows. This avoids an additional divergence penalty on unused latent coefficients under independent augmentation. Our analysis relates approximation accuracy to mixture size, support coverage and dependence within supports, and establishes contraction, selection consistency and a Bernstein-von Mises approximation under explicit conditions on the prior, posterior concentration and variational error. On all 250 simulated datasets with exact posterior references, mixtures reduce errors in inclusion probabilities, grouped support probabilities and coefficient covariance relative to multistart mean field. Comparisons at fixed mixture size and common initialization favor direct joint refinement over augmented or restricted refinement in posterior divergence. Complete stagewise fitting can nevertheless be more accurate near collinearity. The results support direct joint refinement for posterior approximation while showing that local gains do not ensure superiority of the full adaptive search.
ER -