Priors and parameterizations for Bayesian regression with categorical variables and interactions
Abstract
Regression analysis commonly features categorical variables and their interactions to model heterogeneous, group-specific effects. Such interactions are notorious for weak signals and cumbersome parameterizations, and thus urgently require shrinkage. However, the overwhelming default for linear models--using dummy coding, removing a reference group, and applying shrinkage--leads to unwelcome biases, asymmetries, and parameter incongruities across models. We resolve these limitations with a new parameterization-prior combination that couples abundance-based constraints for identifiability with customized shrinkage priors for categorical effects and interactions (BayesABCs). BayesABCs offers a surprising and unique invariance property: the joint posterior of all main effects is (nearly) unchanged by the addition of categorical interactions. We show how this, in conjunction with our shrinkage priors, effectively removes the principal drawbacks of including categorical interactions, thereby making heterogeneous effect estimation broadly feasible. We enforce the identifiability constraints via conditioning and provide a model representation that facilitates efficient posterior computing and direct analysis. Across simulations with a variety of categorical effects and interactions, BayesABCs consistently produces more accurate estimates and predictions, sharper inferences, and unmatched robustness to extraneous interactions relative to competing parameterizations and priors. We apply BayesABCs to study disparities in the error rates of automatic speech recognition systems.
Related
- The Effect of Omitted Variables on the Sign of Regression Coefficients
- Linked-Tucker Factorized Individualized Regression for Paired Multivariate Categorical Outcomes
- Distributional Regression for Arbitrary Response Variables Using ReLU Neural Networks
- Uncertainty Quantification of State Variables Trajectories in the Context of Inverse Problems: An Approach from Bayesian Inference and FDA
- Kumaraswamy-Teissier Distribution: Bayesian and Non-Bayesian Inference, Regression Modeling and Climate Application
BibTeX
@misc{kowal2026priors,
title = {{Priors and parameterizations for Bayesian regression with categorical variables and interactions}},
author = {Daniel R. Kowal and Inkoo Lee},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.36017},
eprint = {2610.36017},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.36017},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Priors and parameterizations for Bayesian regression with categorical variables and interactions
AU - Daniel R. Kowal
AU - Inkoo Lee
PY - 2026
DA - 2026/10/26
PB - arXiv
DO - 10.48550/arxiv.2610.36017
UR - https://arxiv.org/abs/2610.36017
AB - Regression analysis commonly features categorical variables and their interactions to model heterogeneous, group-specific effects. Such interactions are notorious for weak signals and cumbersome parameterizations, and thus urgently require shrinkage. However, the overwhelming default for linear models--using dummy coding, removing a reference group, and applying shrinkage--leads to unwelcome biases, asymmetries, and parameter incongruities across models. We resolve these limitations with a new parameterization-prior combination that couples abundance-based constraints for identifiability with customized shrinkage priors for categorical effects and interactions (BayesABCs). BayesABCs offers a surprising and unique invariance property: the joint posterior of all main effects is (nearly) unchanged by the addition of categorical interactions. We show how this, in conjunction with our shrinkage priors, effectively removes the principal drawbacks of including categorical interactions, thereby making heterogeneous effect estimation broadly feasible. We enforce the identifiability constraints via conditioning and provide a model representation that facilitates efficient posterior computing and direct analysis. Across simulations with a variety of categorical effects and interactions, BayesABCs consistently produces more accurate estimates and predictions, sharper inferences, and unmatched robustness to extraneous interactions relative to competing parameterizations and priors. We apply BayesABCs to study disparities in the error rates of automatic speech recognition systems.
ER -