The Lambert Penalty: Logarithmic Shrinkage for Sparse Regression
Abstract
Sparse regression must balance prediction accuracy with reproducible variable selection. We introduce the Lambert penalty by limiting how quickly the retained fraction of a scalar score increases after variable entry. Maximizing retention under this scale-equivariant constraint yields a logarithmic transition from exact zero to an identity tail. The bounded penalty has a fixed shape and a single penalty parameter selected by training-only five-fold cross-validation. We derive an explicit scalar update, a sharp weak-convexity bound, a sufficient condition for multivariate uniqueness, and conditional convergence of exact cyclic coordinate descent. For fixed dimension, global minimizers are uniformly equivalent to the oracle estimator over an asymptotic tuning band. Gaussian simulations compare Lambert with LASSO, ridge, elastic net, MCP, and SCAD. Lambert favors prediction for strong equal signals and usually improves support recovery over SCAD; MCP often gives better support recovery, and convex methods remain competitive with weak signals or strong correlation. In a QSAR toxicity application, repeated nested cross-validation gives competitive prediction with compact, stable models, although error differences from MCP and SCAD are small. The empirical design followed exploratory development; independent confirmation remains necessary.
Related
- Choosing the penalty in nonparametric regression: short and long-range dependence
- Weak Factor Recovery in Sparse Bayesian Factor Models under Exchangeable Shrinkage Process Prior
- Shrinkage invalidates the Hosmer-Lemeshow test: goodness of fit for penalized logistic regression, with an application to glaucoma diagnosis
- Max-Test-Calibrated Stein Shrinkage with Honest Submodel Selection in Ultra-High-Dimensional Regression
- Byzantine-Robust Sparse Regression with Missing Covariates
BibTeX
@misc{yuzbası2026lambert,
title = {{The Lambert Penalty: Logarithmic Shrinkage for Sparse Regression}},
author = {Bahadır Yüzbaşı},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.09627},
eprint = {2610.09627},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.09627},
}
RIS
TY - UNPB
TI - The Lambert Penalty: Logarithmic Shrinkage for Sparse Regression
AU - Bahadır Yüzbaşı
PY - 2026
DA - 2026/10/07
PB - arXiv
DO - 10.48550/arxiv.2610.09627
UR - https://arxiv.org/abs/2610.09627
AB - Sparse regression must balance prediction accuracy with reproducible variable selection. We introduce the Lambert penalty by limiting how quickly the retained fraction of a scalar score increases after variable entry. Maximizing retention under this scale-equivariant constraint yields a logarithmic transition from exact zero to an identity tail. The bounded penalty has a fixed shape and a single penalty parameter selected by training-only five-fold cross-validation. We derive an explicit scalar update, a sharp weak-convexity bound, a sufficient condition for multivariate uniqueness, and conditional convergence of exact cyclic coordinate descent. For fixed dimension, global minimizers are uniformly equivalent to the oracle estimator over an asymptotic tuning band. Gaussian simulations compare Lambert with LASSO, ridge, elastic net, MCP, and SCAD. Lambert favors prediction for strong equal signals and usually improves support recovery over SCAD; MCP often gives better support recovery, and convex methods remain competitive with weak signals or strong correlation. In a QSAR toxicity application, repeated nested cross-validation gives competitive prediction with compact, stable models, although error differences from MCP and SCAD are small. The empirical design followed exploratory development; independent confirmation remains necessary.
ER -