Distributionally Robust Federated Learning with Multi-Source Data
Abstract
Federated learning trains a shared model from private client data. In practice, data-generating distributions may differ, and the true mixture across clients is often unknown, making the underlying group distribution difficult to specify. Existing approaches address cross-client mixture uncertainty by optimizing against the worst-case mixture, yet assume accurate client-wise distribution estimates. However, these estimates can be unreliable when based on finite samples. To handle both cross-client mixture uncertainty and within-client distributional ambiguity, we construct a global ambiguity set as the union of admissible mixtures of local ambiguity sets. The construction allows client-specific ambiguity radii and admits a client-wise separable reformulation. Leveraging this structure, we establish a high-probability out-of-sample performance guarantee. We further develop a federated algorithm for a penalty-based reformulation and prove its convergence under milder regularity conditions. Simulations validate the algorithm's effectiveness.
Related
- Private and Distributionally Robust Federated Multiclass Classification
- From Dual Tracking to Clipping: Provably Faster Distributionally Robust Multi-Objective Optimization
- Learning Distributionally Robust First-Order Methods for Convex Optimization
- Distributionally Robust Average-Reward Reinforcement Learning: Finite-Sample Guarantees under Weak Communication
- Robust and Sparse Personalized Federated Learning via Gamma-Divergence
BibTeX
@misc{liu2026distributionally,
title = {{Distributionally Robust Federated Learning with Multi-Source Data}},
author = {Yingzhu Liu and Zhongkui Li and Pengcheng You and Ashish Cherukuri},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.20501},
eprint = {2609.20501},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.20501},
}
RIS
TY - UNPB
TI - Distributionally Robust Federated Learning with Multi-Source Data
AU - Yingzhu Liu
AU - Zhongkui Li
AU - Pengcheng You
AU - Ashish Cherukuri
PY - 2026
DA - 2026/09/17
PB - arXiv
DO - 10.48550/arxiv.2609.20501
UR - https://arxiv.org/abs/2609.20501
AB - Federated learning trains a shared model from private client data. In practice, data-generating distributions may differ, and the true mixture across clients is often unknown, making the underlying group distribution difficult to specify. Existing approaches address cross-client mixture uncertainty by optimizing against the worst-case mixture, yet assume accurate client-wise distribution estimates. However, these estimates can be unreliable when based on finite samples. To handle both cross-client mixture uncertainty and within-client distributional ambiguity, we construct a global ambiguity set as the union of admissible mixtures of local ambiguity sets. The construction allows client-specific ambiguity radii and admits a client-wise separable reformulation. Leveraging this structure, we establish a high-probability out-of-sample performance guarantee. We further develop a federated algorithm for a penalty-based reformulation and prove its convergence under milder regularity conditions. Simulations validate the algorithm's effectiveness.
ER -