Generalization Analysis of Distributed Kernel-based Robust Gradient Descent Algorithms
Abstract
In this paper, we investigate the generalization performance of distributed gradient descent algorithms in a reproducing kernel Hilbert space under a robust loss function lσ. By exploiting the spectral characterization of gradient descent together with the intrinsic properties of robust loss functions, we establish optimal learning rates for the distributed kernel-based robust gradient descent (DKRGD) algorithm with an appropriately chosen scale parameter σ. The proposed parameter choice of σ simultaneously alleviates the saturation phenomenon and guarantees statistical robustness. A key technical contribution is a novel error analysis that provides substantially sharper bounds for products of operators, thereby significantly relaxing existing restrictions on the maximum number of local machines while retaining optimal learning rates. Finally, we develop a communication-efficient strategy that further improves the convergence performance of DKRGD.
Related
- A Theoretical Analysis of Generalization Dynamics in Neural Networks under Gradient Descent with Weight Decay
- Overfitting of Spectral Gradient Descent: How Matrix Geometry shapes Generalization and Implicit Bias
- Averaged Mirror Descent and Dual Gradient Methods: Convergent Algorithms for Entropic Gromov-Wasserstein Problems
- Distributed Proximal Stein Variational Gradient Descent Algorithm for Large-scale Bayesian Inference in Traveltime Tomography
- Corrections to ``The Method of Gaps: Exact Expressions for the Generalization Error of Supervised Learning Algorithms
BibTeX
@misc{meng2026generalization,
title = {{Generalization Analysis of Distributed Kernel-based Robust Gradient Descent Algorithms}},
author = {Jun-Yi Meng and Zheng-Chu Guo and Yuan Mao},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.11712},
eprint = {2609.11712},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.11712},
}
RIS
TY - UNPB
TI - Generalization Analysis of Distributed Kernel-based Robust Gradient Descent Algorithms
AU - Jun-Yi Meng
AU - Zheng-Chu Guo
AU - Yuan Mao
PY - 2026
DA - 2026/09/10
PB - arXiv
DO - 10.48550/arxiv.2609.11712
UR - https://arxiv.org/abs/2609.11712
AB - In this paper, we investigate the generalization performance of distributed gradient descent algorithms in a reproducing kernel Hilbert space under a robust loss function l_σ. By exploiting the spectral characterization of gradient descent together with the intrinsic properties of robust loss functions, we establish optimal learning rates for the distributed kernel-based robust gradient descent (DKRGD) algorithm with an appropriately chosen scale parameter σ. The proposed parameter choice of σ simultaneously alleviates the saturation phenomenon and guarantees statistical robustness. A key technical contribution is a novel error analysis that provides substantially sharper bounds for products of operators, thereby significantly relaxing existing restrictions on the maximum number of local machines while retaining optimal learning rates. Finally, we develop a communication-efficient strategy that further improves the convergence performance of DKRGD.
ER -