Representative Learning for Distributed Support Vector Machines
Abstract
Distributed support vector machine (SVM) learning is challenging under heterogeneous local data and limited communication. We develop a Representative Learning framework based on Lifted Empirical Risk Minimization, replacing each covariate by a local Gaussian distribution. For linear SVMs, the resulting expected hinge loss has an exact closed form, is smooth for nondegenerate lifting covariances, and converges to the ordinary hinge loss as the covariance vanishes. The lifted formulation enables clouded representatives that encode both location and covariance. We study a model-independent Mean Representative (MR) and a Score-Matching Representative (SMR) that matches the class-wise lifted gradient at its construction parameter. We show that SMR covariance scaling is stable with respect to within-group heterogeneity and that the gradient discrepancy between full data and frozen representatives is controlled by the maximum within-group diameter. Under strong convexity and smoothness, this yields an inexact-gradient convergence guarantee. Experiments under heterogeneous class allocation show that lifted representatives remain competitive with centralized solutions, while SMR more closely recovers the centralized lifted-SVM optimizer. We further extend the framework to nonlinear SVMs through a shared kernel dictionary, with experiments on Ionosphere and Australian Credit Approval showing analogous behavior. These results provide a distributional representative framework for communication-efficient distributed SVM learning under heterogeneity.
Related
- Support Vector Machines for Classification, Regression, and Outlier Detection: A Tutorial
- Comparative Performance Analysis of Optimization Solvers for Linear Support Vector Machines: Impact of Preconditioning and Dimensionality An Empirical Study Using the Benchopt Framework
- Support Vector Machines Directly Minimise KL Divergence
- High-Dimensional Statistical Inference for Sparse Support Vector Machines
- Asymptotic Properties of Support Vector Machines in High-Dimension, Low-Sample-Size Settings under a Spiked Model
BibTeX
@misc{casey2026representative,
title = {{Representative Learning for Distributed Support Vector Machines}},
author = {Joseph A. Casey and Boxiang Wang and Keren Li},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.00641},
eprint = {2611.00641},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.00641},
}
RIS
TY - UNPB
TI - Representative Learning for Distributed Support Vector Machines
AU - Joseph A. Casey
AU - Boxiang Wang
AU - Keren Li
PY - 2026
DA - 2026/11/01
PB - arXiv
DO - 10.48550/arxiv.2611.00641
UR - https://arxiv.org/abs/2611.00641
AB - Distributed support vector machine (SVM) learning is challenging under heterogeneous local data and limited communication. We develop a Representative Learning framework based on Lifted Empirical Risk Minimization, replacing each covariate by a local Gaussian distribution. For linear SVMs, the resulting expected hinge loss has an exact closed form, is smooth for nondegenerate lifting covariances, and converges to the ordinary hinge loss as the covariance vanishes. The lifted formulation enables clouded representatives that encode both location and covariance. We study a model-independent Mean Representative (MR) and a Score-Matching Representative (SMR) that matches the class-wise lifted gradient at its construction parameter. We show that SMR covariance scaling is stable with respect to within-group heterogeneity and that the gradient discrepancy between full data and frozen representatives is controlled by the maximum within-group diameter. Under strong convexity and smoothness, this yields an inexact-gradient convergence guarantee. Experiments under heterogeneous class allocation show that lifted representatives remain competitive with centralized solutions, while SMR more closely recovers the centralized lifted-SVM optimizer. We further extend the framework to nonlinear SVMs through a shared kernel dictionary, with experiments on Ionosphere and Australian Credit Approval showing analogous behavior. These results provide a distributional representative framework for communication-efficient distributed SVM learning under heterogeneity.
ER -