Cost-based variable selection for prediction models
Abstract
Objective: Variable selection for clinical prediction models is commonly done on statistical grounds related to model fit. Statistical approaches ignore the cost of misclassifications and the harm of measuring a variable. We aim to develop a decision-analytic variable selection algorithm based on net benefit. Materials and Methods: The proposed algorithm selects the set of predictors by exhaustively comparing the net benefit of all combinations using cross validation, with an extension based on backward elimination for settings with many predictors. Variables are selected if the increased net benefit from better predictiveness exceeds the net benefit reduction from the harm of variable measurement. Harm includes financial expense, time, resource allocation, and patient burden. We illustrate the algorithm through a simulated example and three real case studies. We compare its performance to traditional backward elimination and LASSO. Results: Across the case studies, parsimonious models were generated that optimized clinical utility with minimal impact on discrimination and calibration. Penalizing predictors based on their cost ensured that expensive or burdensome variables were only selected if they offer sufficient clinical utility. The algorithm was implemented in the R package NBvarsel. Discussion: Decision-analytic variable selection allows prediction models to maximize patient benefit while minimizing unnecessary clinical burden and costs. Conclusion: Clinical utility should not be an afterthought reserved only for implementation. By integrating a harm-adjusted net benefit metric directly into the variable selection process, NBvarsel allows researchers to select only the variables that sufficiently improve clinical utility.
Related
- Non-Fragile Guaranteed Cost Control for Uncertain Fractional Delay Systems of Variable Order with Time-Varying Control Input Delays: A Possibly Stability Analysis
- Prior-data fitted networks for variable selection
- Local Epochs, Averaging, and Variable Selection in Federated Lasso
- Co-sufficient weighting: leveraging sparsity for more powerful variable selection in the linear model
- Robust Dual-Regularized Variable Selection under Outlier Contamination
BibTeX
@misc{barrenada2026costbased,
title = {{Cost-based variable selection for prediction models}},
author = {Lasai Barreñada and Andrew Vickers and Laure Wynants and Bieke Tack and Dirk Timmerman and Ewout Steyerberg and Ben Van Calster},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.37353},
eprint = {2610.37353},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.37353},
note = {CC BY-NC-SA 4.0},
}
RIS
TY - UNPB
TI - Cost-based variable selection for prediction models
AU - Lasai Barreñada
AU - Andrew Vickers
AU - Laure Wynants
AU - Bieke Tack
AU - Dirk Timmerman
AU - Ewout Steyerberg
AU - Ben Van Calster
PY - 2026
DA - 2026/10/27
PB - arXiv
DO - 10.48550/arxiv.2610.37353
UR - https://arxiv.org/abs/2610.37353
AB - Objective: Variable selection for clinical prediction models is commonly done on statistical grounds related to model fit. Statistical approaches ignore the cost of misclassifications and the harm of measuring a variable. We aim to develop a decision-analytic variable selection algorithm based on net benefit. Materials and Methods: The proposed algorithm selects the set of predictors by exhaustively comparing the net benefit of all combinations using cross validation, with an extension based on backward elimination for settings with many predictors. Variables are selected if the increased net benefit from better predictiveness exceeds the net benefit reduction from the harm of variable measurement. Harm includes financial expense, time, resource allocation, and patient burden. We illustrate the algorithm through a simulated example and three real case studies. We compare its performance to traditional backward elimination and LASSO. Results: Across the case studies, parsimonious models were generated that optimized clinical utility with minimal impact on discrimination and calibration. Penalizing predictors based on their cost ensured that expensive or burdensome variables were only selected if they offer sufficient clinical utility. The algorithm was implemented in the R package NBvarsel. Discussion: Decision-analytic variable selection allows prediction models to maximize patient benefit while minimizing unnecessary clinical burden and costs. Conclusion: Clinical utility should not be an afterthought reserved only for implementation. By integrating a harm-adjusted net benefit metric directly into the variable selection process, NBvarsel allows researchers to select only the variables that sufficiently improve clinical utility.
ER -