High-Dimensional Statistical Inference for Sparse Support Vector Machines
Abstract
Using a replica-symmetric high-dimensional characterization, we develop an inferential framework for sparse support vector machines when the sample size and number of features grow proportionally. The main challenge is the nonsmooth hinge loss, which prevents direct application of debiasing arguments developed for smooth classification losses. We overcome this difficulty by representing the L1-penalized support vector machine (SVM) as a linear program and identifying the hinge-loss subgradient through its dual variables. This yields a computationally accessible debiased estimator whose coordinates are asymptotically Gaussian under the proportional asymptotic regime. The resulting distributional characterization provides confidence intervals and hypothesis tests for individual features and enables false-discovery-rate-controlled variable selection. Extensive simulations examine calibration, power, and variable-selection performance under a range of covariance structures, including strongly correlated designs. An analysis of high-dimensional breast cancer gene-expression data illustrates how the proposed inference can distinguish statistically significant features from variables selected by the original sparse SVM.
Related
- Support Vector Machines for Classification, Regression, and Outlier Detection: A Tutorial
- Comparative Performance Analysis of Optimization Solvers for Linear Support Vector Machines: Impact of Preconditioning and Dimensionality An Empirical Study Using the Benchopt Framework
- Support Vector Machines Directly Minimise KL Divergence
- Bayesian Tensor Train Kernel Machines with Automatic Model Complexity Inference
- Representative Learning for Distributed Support Vector Machines
BibTeX
@misc{zeng2026highdimensional,
title = {{High-Dimensional Statistical Inference for Sparse Support Vector Machines}},
author = {Peng Zeng and Hanwen Huang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.08345},
eprint = {2610.08345},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.08345},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - High-Dimensional Statistical Inference for Sparse Support Vector Machines
AU - Peng Zeng
AU - Hanwen Huang
PY - 2026
DA - 2026/10/06
PB - arXiv
DO - 10.48550/arxiv.2610.08345
UR - https://arxiv.org/abs/2610.08345
AB - Using a replica-symmetric high-dimensional characterization, we develop an inferential framework for sparse support vector machines when the sample size and number of features grow proportionally. The main challenge is the nonsmooth hinge loss, which prevents direct application of debiasing arguments developed for smooth classification losses. We overcome this difficulty by representing the L_1-penalized support vector machine (SVM) as a linear program and identifying the hinge-loss subgradient through its dual variables. This yields a computationally accessible debiased estimator whose coordinates are asymptotically Gaussian under the proportional asymptotic regime. The resulting distributional characterization provides confidence intervals and hypothesis tests for individual features and enables false-discovery-rate-controlled variable selection. Extensive simulations examine calibration, power, and variable-selection performance under a range of covariance structures, including strongly correlated designs. An analysis of high-dimensional breast cancer gene-expression data illustrates how the proposed inference can distinguish statistically significant features from variables selected by the original sparse SVM.
ER -