Component-weighted sparse group LASSO for finite mixture regression models
Abstract
When analyzing heterogeneous data with latent subpopulations, finite mixture regression (FMR) models are effective as they allow for variations in regression coefficients across mixture components. Variable selection is important at two levels: (1) at the group level by removing completely irrelevant covariates across all subpopulations and (2) at the individual level by removing irrelevant covariates within each subpopulation. However, existing variable selection methods for FMR focus solely on individual-level regression coefficient selection, providing no mechanism for group-level variable elimination. We introduce a component-weighted sparse group LASSO regularization method that performs efficient variable selection at both levels simultaneously in finite mixture regression models belonging to the exponential family. We develop a novel optimization procedure through a Majorization-Minimization algorithm. Simulation studies under Gaussian and Poisson mixture regression settings demonstrate reliable recovery of the true sparsity structure at both levels. The method is illustrated by analyzing two real datasets.
Related
- Bootstrap Confidence Sets for the Number of Components in Finite Mixture Regression Models
- Identifiability from Moments in Independent Component Analysis Mixture Models
- When Does Sparse Routing Help? The Geometry of Expert Separation in Mixture of Experts
- Server-Side Sparse Assignment for Federated Mixture-of-Experts
- Sparsity Saves Compute, Not Samples: Oracle Inequalities for Dense and Sparse Mixture of Experts
BibTeX
@misc{joshy2026componentweighted,
title = {{Component-weighted sparse group LASSO for finite mixture regression models}},
author = {Vinay Joshy and Zeny Feng and Grace Stelter and Lorna E Deeth and Alysha Cooper},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.16050},
eprint = {2609.16050},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.16050},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Component-weighted sparse group LASSO for finite mixture regression models
AU - Vinay Joshy
AU - Zeny Feng
AU - Grace Stelter
AU - Lorna E Deeth
AU - Alysha Cooper
PY - 2026
DA - 2026/09/12
PB - arXiv
DO - 10.48550/arxiv.2609.16050
UR - https://arxiv.org/abs/2609.16050
AB - When analyzing heterogeneous data with latent subpopulations, finite mixture regression (FMR) models are effective as they allow for variations in regression coefficients across mixture components. Variable selection is important at two levels: (1) at the group level by removing completely irrelevant covariates across all subpopulations and (2) at the individual level by removing irrelevant covariates within each subpopulation. However, existing variable selection methods for FMR focus solely on individual-level regression coefficient selection, providing no mechanism for group-level variable elimination. We introduce a component-weighted sparse group LASSO regularization method that performs efficient variable selection at both levels simultaneously in finite mixture regression models belonging to the exponential family. We develop a novel optimization procedure through a Majorization-Minimization algorithm. Simulation studies under Gaussian and Poisson mixture regression settings demonstrate reliable recovery of the true sparsity structure at both levels. The method is illustrated by analyzing two real datasets.
ER -