Robust LassoNet: Enhancing Feature Selection in Neural Networks via Robust Loss Functions
Abstract
Feature selection in neural networks remains a challenging problem, particularly in the presence of noisy or contaminated data. LassoNet is a recent approach that addresses this issue by combining neural networks with hierarchical sparsity constraints, enabling simultaneous prediction and variable selection. However, its standard formulation relies on the mean squared error (MSE) loss, which is known to be highly sensitive to outliers. In this paper, we present Robust LassoNet, an extension of LassoNet that incorporates robust loss functions, such as Huber, Cauchy, Tukey's bisquare, and Nonnegative Garrote, to mitigate the effect of extreme observations. The proposed approach preserves the original optimization framework while improving stability under data contamination. Through experiments on synthetic and real datasets, we show that robust LassoNet significantly improves both predictive performance and feature selection accuracy in the presence of outliers or heavy tailed noise, while maintaining comparable performance in clean settings. These results highlight the importance of robustness in neural network based feature selection and suggest practical guidelines for choosing appropriate loss functions.
Related
- Learning from pandemics: Feature selection and clustering for robust digital disease nowcasting
- Universal Feature Selection with Noisy Observations and Weak Symmetry Conditions
- Variable Selection for Feature-Based Newsvendor
- HLR-UFS: Hessian-based Unsupervised Feature Selection using Low-Rank Approximation
- Efficient Benchmarking Is Just Feature Selection and Multiple Regression
BibTeX
@misc{canditiis2026robust,
title = {{Robust LassoNet: Enhancing Feature Selection in Neural Networks via Robust Loss Functions}},
author = {Daniela De Canditiis and Italia De Feis and Paola Stolfi},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2609.38263},
eprint = {2609.38263},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.38263},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Robust LassoNet: Enhancing Feature Selection in Neural Networks via Robust Loss Functions
AU - Daniela De Canditiis
AU - Italia De Feis
AU - Paola Stolfi
PY - 2026
DA - 2026/10/09
PB - arXiv
DO - 10.48550/arxiv.2609.38263
UR - https://arxiv.org/abs/2609.38263
AB - Feature selection in neural networks remains a challenging problem, particularly in the presence of noisy or contaminated data. LassoNet is a recent approach that addresses this issue by combining neural networks with hierarchical sparsity constraints, enabling simultaneous prediction and variable selection. However, its standard formulation relies on the mean squared error (MSE) loss, which is known to be highly sensitive to outliers. In this paper, we present Robust LassoNet, an extension of LassoNet that incorporates robust loss functions, such as Huber, Cauchy, Tukey's bisquare, and Nonnegative Garrote, to mitigate the effect of extreme observations. The proposed approach preserves the original optimization framework while improving stability under data contamination. Through experiments on synthetic and real datasets, we show that robust LassoNet significantly improves both predictive performance and feature selection accuracy in the presence of outliers or heavy tailed noise, while maintaining comparable performance in clean settings. These results highlight the importance of robustness in neural network based feature selection and suggest practical guidelines for choosing appropriate loss functions.
ER -