A knowledge-driven nonconvex square-root regularization framework for joint group sparsity and structured smoothness
Abstract
Structured sparse learning is a leading paradigm for high-dimensional data modeling, as it enables rigorous integration of domain-validated structural priors into the learning process. Among these priors, pre-defined grouped structure and piecewise smoothness are ubiquitous in real-world applications. However, most existing structured regularization methods are built on squared loss and convex penalties, rarely jointly model these two complementary structural priors, and suffer from two key limitations. Their optimal tuning parameters are sensitive to the unknown noise level σ due to the squared loss. Meanwhile, convex penalties induce inherent estimation bias and severe over-shrinkage of large magnitudes. To address these issues, we propose a knowledge-driven Nonconvex Structured Smooth Adjustment for Square-Root Regularization (NS3R), which unifies the σ-free square-root loss with a joint nonconvex ℓ2,q penalty (0 < q < 1) to simultaneously encode group sparsity and structured smoothness priors. Theoretically, we establish rigorous non-asymptotic error bounds for the NS3R estimator under a mild (2,q)-group restricted compatibility condition. Computationally, we develop an efficient double-screened IRLS with thresholding (DS-IRLS-th) algorithm, which adopts a nested double-screening strategy to reduce computational cost and improve numerical stability, with provable global convergence for the resulting nonconvex optimization problem. Extensive experiments on synthetic and real-world data demonstrate that NS3R consistently outperforms state-of-the-art methods in finite-sample settings.
Related
- Dense Weak Hiding: Closing Complexity Gaps in Nonconvex and PL Finite-Sum Optimization under Individual Smoothness
- On the Regularization of Primal-Dual Dynamics in Time-Dependent Nonconvex Optimization
- Lower- and upper-C2 integrands: a structured class of functions for stochastic nonconvex optimization
- Preconditioned Three-Term Conjugate Gradient Framework for Nonconvex Finite Minimax Problems
- A Nonconvex Framework for Joint Sparse Sample Weighting and Parameter Estimation
BibTeX
@misc{xie2026knowledgedriven,
title = {{A knowledge-driven nonconvex square-root regularization framework for joint group sparsity and structured smoothness}},
author = {Wanling Xie and Hu Yang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.35428},
eprint = {2610.35428},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.35428},
}
RIS
TY - UNPB
TI - A knowledge-driven nonconvex square-root regularization framework for joint group sparsity and structured smoothness
AU - Wanling Xie
AU - Hu Yang
PY - 2026
DA - 2026/10/26
PB - arXiv
DO - 10.48550/arxiv.2610.35428
UR - https://arxiv.org/abs/2610.35428
AB - Structured sparse learning is a leading paradigm for high-dimensional data modeling, as it enables rigorous integration of domain-validated structural priors into the learning process. Among these priors, pre-defined grouped structure and piecewise smoothness are ubiquitous in real-world applications. However, most existing structured regularization methods are built on squared loss and convex penalties, rarely jointly model these two complementary structural priors, and suffer from two key limitations. Their optimal tuning parameters are sensitive to the unknown noise level σ due to the squared loss. Meanwhile, convex penalties induce inherent estimation bias and severe over-shrinkage of large magnitudes. To address these issues, we propose a knowledge-driven Nonconvex Structured Smooth Adjustment for Square-Root Regularization (NS^3R), which unifies the σ-free square-root loss with a joint nonconvex ℓ_2,q penalty (0 < q < 1) to simultaneously encode group sparsity and structured smoothness priors. Theoretically, we establish rigorous non-asymptotic error bounds for the NS^3R estimator under a mild (2,q)-group restricted compatibility condition. Computationally, we develop an efficient double-screened IRLS with thresholding (DS-IRLS-th) algorithm, which adopts a nested double-screening strategy to reduce computational cost and improve numerical stability, with provable global convergence for the resulting nonconvex optimization problem. Extensive experiments on synthetic and real-world data demonstrate that NS^3R consistently outperforms state-of-the-art methods in finite-sample settings.
ER -