Square-Root Clustered Lasso for High-Dimensional Linear Regression with Finite-Sample Theory and Efficient Algorithm Implementation
Abstract
This paper proposes a square-root clustered Lasso penalized method for high-dimensional linear regression. By integrating square-root loss with the clustered Lasso penalty, the resulting method enables flexible regression modeling and allows tuning parameter selection independent of the unknown noise level. We establish weak decomposability of the clustered Lasso penalty and a noise-level-free tuning strategy under Gaussian noise. Furthermore, we establish the finite-sample prediction accuracy guarantees for the proposed method. The inherent nonsmoothness of the square-root loss and clustered Lasso penalty poses considerable computational difficulties. To resolve this issue, we develop an efficient semismooth Newton-based proximal augmented Lagrangian algorithm (N-ALM) to solve the corresponding dual optimization problem. The subproblems arising in the iteration are efficiently tackled by the semismooth Newton method, which fully utilizes the sparsity structure of the generalized Jacobian. We further establish global convergence and a local linear rate for the distance to the KKT solution set under suitable conditions; the local rate becomes superlinear when the penalty parameter tends to infinity. We conduct numerical experiments on real-world datasets to evaluate the computational performance of N-ALM. The results demonstrate that N-ALM solves the tested high-dimensional SRCL problems accurately and substantially reduces running times compared with d-ADMM.
Related
- SuperPCA: subspace analysis and an efficient algorithm for high-dimensional PCA
- Cluster-induced target shift and synthetic approximation for high-dimensional clustered data
- Efficient estimation methods for non-Gaussian regression models in continuous time
- A Stochastic EM Algorithm with Sampling-Importance Resampling for Missing Data in Regression with Nonlinear Predictors
- Optimization of Quadratic Sieve Algorithm Implementation for Large Integer Factorization
BibTeX
@misc{wang2026squareroot,
title = {{Square-Root Clustered Lasso for High-Dimensional Linear Regression with Finite-Sample Theory and Efficient Algorithm Implementation}},
author = {Shuqi Wang and Yunhai Xiao and Mengjiao Shi},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.35390},
eprint = {2610.35390},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.35390},
}
RIS
TY - UNPB
TI - Square-Root Clustered Lasso for High-Dimensional Linear Regression with Finite-Sample Theory and Efficient Algorithm Implementation
AU - Shuqi Wang
AU - Yunhai Xiao
AU - Mengjiao Shi
PY - 2026
DA - 2026/10/26
PB - arXiv
DO - 10.48550/arxiv.2610.35390
UR - https://arxiv.org/abs/2610.35390
AB - This paper proposes a square-root clustered Lasso penalized method for high-dimensional linear regression. By integrating square-root loss with the clustered Lasso penalty, the resulting method enables flexible regression modeling and allows tuning parameter selection independent of the unknown noise level. We establish weak decomposability of the clustered Lasso penalty and a noise-level-free tuning strategy under Gaussian noise. Furthermore, we establish the finite-sample prediction accuracy guarantees for the proposed method. The inherent nonsmoothness of the square-root loss and clustered Lasso penalty poses considerable computational difficulties. To resolve this issue, we develop an efficient semismooth Newton-based proximal augmented Lagrangian algorithm (N-ALM) to solve the corresponding dual optimization problem. The subproblems arising in the iteration are efficiently tackled by the semismooth Newton method, which fully utilizes the sparsity structure of the generalized Jacobian. We further establish global convergence and a local linear rate for the distance to the KKT solution set under suitable conditions; the local rate becomes superlinear when the penalty parameter tends to infinity. We conduct numerical experiments on real-world datasets to evaluate the computational performance of N-ALM. The results demonstrate that N-ALM solves the tested high-dimensional SRCL problems accurately and substantially reduces running times compared with d-ADMM.
ER -