Conjugate priors for Bayesian Variable Selection in exponential family Generalized Linear Models
Abstract
Covariate selection in Generalized Linear Models (GLMs) is a fundamental problem in statistics, as including irrelevant predictors might lead to overfitting and poor interpretability, while omitting relevant ones might result in biased estimates. Most Bayesian approaches to variable selection -- including spike-and-slab priors and continuous shrinkage priors -- have key limitations, e.g., (i) are based on non fully conjugate formulations, (ii) are restricted to a linear model, or (iii) lack posterior consistency guarantees for the variable selection procedure and model parameters. In this work, we propose a fully Bayesian hierarchical and conjugate framework for covariate selection in GLMs, applicable to any distribution in the exponential family, based on modeling a binary inclusion indicator that directly encodes covariate inclusion in the linear predictor. In our approach, variable selection and parameter estimation are performed simultaneously, incorporating both sources of uncertainty in posterior inference. Consequently, our methodology provides a valid post-model Bayesian selection procedure. We present theoretical guarantees of the proposed fully conjugate Bayesian variable selection for GLMs, establishing posterior consistency of both the inclusion indicators and the active regression coefficients. We derive an efficient Gibbs Sampling algorithm with a corresponding R package implementation. We validate the proposed method on synthetic and real-world datasets, demonstrating competitive predictive and inferential performance.
Related
- Bayesian Variable Selection for High-Dimensional Predictors with Missing Psychometric Outcomes
- A Bayesian Bi-Directional Splitting Framework for Variable Selection in Large Datasets
- FDR-Controlled Variable Selection for Generalized Linear Models and Cox Regression with Virtual Dummies
- Refining Relational Event Models: Bayesian Penalization and Variable Selection in REMs
- Accelerating Bayesian Variable Selection using Piecewise Deterministic Markov Processes
BibTeX
@misc{filippozzi2026conjugate,
title = {{Conjugate priors for Bayesian Variable Selection in exponential family Generalized Linear Models}},
author = {Lucia Filippozzi and Iñigo Urteaga and Claudio Agostinelli},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2606.24357},
eprint = {2606.24357},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.24357},
}
RIS
TY - UNPB
TI - Conjugate priors for Bayesian Variable Selection in exponential family Generalized Linear Models
AU - Lucia Filippozzi
AU - Iñigo Urteaga
AU - Claudio Agostinelli
PY - 2026
DA - 2026/11/04
PB - arXiv
DO - 10.48550/arxiv.2606.24357
UR - https://arxiv.org/abs/2606.24357
AB - Covariate selection in Generalized Linear Models (GLMs) is a fundamental problem in statistics, as including irrelevant predictors might lead to overfitting and poor interpretability, while omitting relevant ones might result in biased estimates. Most Bayesian approaches to variable selection -- including spike-and-slab priors and continuous shrinkage priors -- have key limitations, e.g., (i) are based on non fully conjugate formulations, (ii) are restricted to a linear model, or (iii) lack posterior consistency guarantees for the variable selection procedure and model parameters. In this work, we propose a fully Bayesian hierarchical and conjugate framework for covariate selection in GLMs, applicable to any distribution in the exponential family, based on modeling a binary inclusion indicator that directly encodes covariate inclusion in the linear predictor. In our approach, variable selection and parameter estimation are performed simultaneously, incorporating both sources of uncertainty in posterior inference. Consequently, our methodology provides a valid post-model Bayesian selection procedure. We present theoretical guarantees of the proposed fully conjugate Bayesian variable selection for GLMs, establishing posterior consistency of both the inclusion indicators and the active regression coefficients. We derive an efficient Gibbs Sampling algorithm with a corresponding R package implementation. We validate the proposed method on synthetic and real-world datasets, demonstrating competitive predictive and inferential performance.
ER -