Comparing the use of supervised machine learning variable selection methods in the context of two-group classification in the psychological and health sciences
7 affiliations
- 1Department of Psychology, California State University Northridge
- 2Department of Psychology, University of Oklahoma
- 3School of Psychological and Brain Sciences, Georgia Institute of Technology
- 4School of Electrical and Computer Engineering, University of Oklahoma
- 5Center on Alcohol, Substance use, And Addictions, University of New Mexico
- 6Center on Alcohol, Substance Use, and Addictions, University of New Mexico
- 7Department of Psychology, University of New Mexico
Abstract
Introduction: Variable selection (VS) is crucial for building accurate and generalizable classification models. Reducing the necessary number of variables improves model efficiency, interpretability, and generalizability while reducing data collection burden. Despite the availability of various VS methods, their comparative performance within machine learning frameworks for classification remains unclear. Methods: This study conducted a large-scale Monte Carlo simulation to compare VS methods from distinct families: regularization techniques (LASSO, Elastic Net), tree-based methods (random forest-based Boruta wrapper), penalized support vector machines, and metaheuristics (genetic algorithm). An empirical example from an alcohol use disorder dataset illustrated the comparative findings. Supplementary simulations evaluated four additional filter methods, re-estimated performance under stratified 10-fold cross-validation, and quantified overfitting via train-test performance gaps. Results: In the main simulation, the genetic algorithm demonstrated the strongest overall classification accuracy, while LASSO offered the most parsimonious solution with minimal performance loss. All three top-performing methods (GA, LASSO, Elastic Net) substantially reduced the number of variables from the saturated model. Supplementary analyses aligned with main simulation findings. Discussion: These findings highlight the practical value of machine learning VS methods in psychological research by demonstrating how to balance accuracy, interpretability, and scalability. The results provide guidance for researchers selecting among competing VS approaches based on their priorities regarding model performance versus parsimony.
Related
- Semi-Supervised Classification with Informative Missing Labels in Weibull Mixture Models
- Semi-supervised learning on singular data sets
- Corrections to ``The Method of Gaps: Exact Expressions for the Generalization Error of Supervised Learning Algorithms
- Neural Supervised Clustering for Learning Individualized Treatment Rules with Many Treatments
- Rethinking Representation Geometry in Supervised Contrastive Learning under Long-Tailed Data
BibTeX
@article{bain2026comparing,
title = {{Comparing the use of supervised machine learning variable selection methods in the context of two-group classification in the psychological and health sciences}},
author = {Catherine M. Bain and Dingjing Shi and Yaser M. Banad and Cassandra L. Boness and Jordan E. Loeffelman},
journal = {Frontiers in Psychology},
year = {2026},
month = sep,
doi = {10.3389/fpsyg.2026.1819366},
url = {https://doi.org/10.3389/fpsyg.2026.1819366},
note = {CC BY},
}
RIS
TY - JOUR
TI - Comparing the use of supervised machine learning variable selection methods in the context of two-group classification in the psychological and health sciences
AU - Catherine M. Bain
AU - Dingjing Shi
AU - Yaser M. Banad
AU - Cassandra L. Boness
AU - Jordan E. Loeffelman
PY - 2026
DA - 2026/09/07
JO - Frontiers in Psychology
DO - 10.3389/fpsyg.2026.1819366
UR - https://doi.org/10.3389/fpsyg.2026.1819366
AB - Introduction: Variable selection (VS) is crucial for building accurate and generalizable classification models. Reducing the necessary number of variables improves model efficiency, interpretability, and generalizability while reducing data collection burden. Despite the availability of various VS methods, their comparative performance within machine learning frameworks for classification remains unclear. Methods: This study conducted a large-scale Monte Carlo simulation to compare VS methods from distinct families: regularization techniques (LASSO, Elastic Net), tree-based methods (random forest-based Boruta wrapper), penalized support vector machines, and metaheuristics (genetic algorithm). An empirical example from an alcohol use disorder dataset illustrated the comparative findings. Supplementary simulations evaluated four additional filter methods, re-estimated performance under stratified 10-fold cross-validation, and quantified overfitting via train-test performance gaps. Results: In the main simulation, the genetic algorithm demonstrated the strongest overall classification accuracy, while LASSO offered the most parsimonious solution with minimal performance loss. All three top-performing methods (GA, LASSO, Elastic Net) substantially reduced the number of variables from the saturated model. Supplementary analyses aligned with main simulation findings. Discussion: These findings highlight the practical value of machine learning VS methods in psychological research by demonstrating how to balance accuracy, interpretability, and scalability. The results provide guidance for researchers selecting among competing VS approaches based on their priorities regarding model performance versus parsimony.
SN - 1664-1078
ER -