Adaptive Federated Optimization Across Smooth and Nonsmooth Regimes
Abstract
AdaGrad-style methods can simultaneously adapt to both the order and constants of smoothness in centralized optimization. However, existing extensions to federated optimization with heterogeneous client objectives and multiple local steps require the smoothness order to be specified in advance. We address this limitation with AdaFed, a federated method that adapts across several regimes under a single choice of parameters. In the deterministic setting, AdaFed achieves an O(1/T) convergence rate for both smooth nonconvex and smooth convex objectives, and an O(1/√T) rate for Lipschitz nonsmooth convex objectives up to logarithmic factors. Under a light-tailed stochastic-gradient noise assumption, these guarantees extend to the stochastic setting with additional logarithmic factors and noise-dependent terms. Experiments across the three problem classes further support our theoretical findings.
Related
- Exact penalty methods with adaptive discretization for nonsmooth semi-infinite optimization
- Linear convergence rate analysis of Proximal-Tracking for nonsmooth distributed optimization
- Discrete-time feedback linearization control for nonsmooth constrained optimization
- The Complexity of Finding Stationary Points in Nonsmooth Nonconvex Optimization
- Proximal Limited-Memory Quasi-Newton Methods for Nonsmooth Nonconvex Optimization
BibTeX
@misc{wang2026adaptive,
title = {{Adaptive Federated Optimization Across Smooth and Nonsmooth Regimes}},
author = {Zimeng Wang and Ali Dadras and Alp Yurtsever},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.39716},
eprint = {2610.39716},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.39716},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Adaptive Federated Optimization Across Smooth and Nonsmooth Regimes
AU - Zimeng Wang
AU - Ali Dadras
AU - Alp Yurtsever
PY - 2026
DA - 2026/10/29
PB - arXiv
DO - 10.48550/arxiv.2610.39716
UR - https://arxiv.org/abs/2610.39716
AB - AdaGrad-style methods can simultaneously adapt to both the order and constants of smoothness in centralized optimization. However, existing extensions to federated optimization with heterogeneous client objectives and multiple local steps require the smoothness order to be specified in advance. We address this limitation with AdaFed, a federated method that adapts across several regimes under a single choice of parameters. In the deterministic setting, AdaFed achieves an O(1/T) convergence rate for both smooth nonconvex and smooth convex objectives, and an O(1/√T) rate for Lipschitz nonsmooth convex objectives up to logarithmic factors. Under a light-tailed stochastic-gradient noise assumption, these guarantees extend to the stochastic setting with additional logarithmic factors and noise-dependent terms. Experiments across the three problem classes further support our theoretical findings.
ER -