Group Distributionally Robust Optimization over Decentralized Networks
Abstract
We study group distributionally robust optimization (GDRO) over decentralized networks, where workers collaborate to learn robust models by minimizing the maximum local risk. Existing methods optimize only dual-regularized surrogates and establish guarantees for a globally averaged model that is not available to workers. In this paper, we directly optimize the original GDRO objective and establish guarantees for local models in semi-decentralized (SemiDec) and fully decentralized (Dec) settings. A key difficulty is that applying the primal-dual decomposition from centralized GDRO yields worker-dependent losses whose evaluation requires unavailable cross-worker information. To address this challenge, we develop a novel error decomposition that recasts decentralized GDRO as (i) a decentralized online convex optimization (D-OCO) problem and a prediction-with-expert-advice problem in SemiDec, and as (ii) two coupled D-OCO problems in Dec, with discrepancy terms capturing inconsistencies among local decisions. By isolating consensus errors in analysis, we develop modular frameworks that support different consensus routines. We then establish the \emph{first} per-worker convergence guarantees for optimizing the original GDRO objective over decentralized networks. Instantiations with accelerated gossip and Choco-gossip achieve an O(√((log m)/T)) convergence rate over T update rounds, while experiments demonstrate improved worst-worker performance over baselines.
Related
- Data-driven joint optimization of maintenance and spare parts provisioning: A distributionally robust approach
- Conformal-DRO: Distributionally Robust Optimization with Conformalized Ambiguity Set
- Calibrating Ambiguity Set via Diagnostic Transport for Distributionally Robust Optimization
- Stochastic Mirror-Prox Optimisation of Group Distributionally Robust Multimodal Logistic Regression for Multilingual Fact-Checking Verdict Prediction
- Higher-Order Sensitivity Analysis of Wasserstein and Adapted Wasserstein Distributionally Robust Optimization Problems
BibTeX
@misc{bai2026group,
title = {{Group Distributionally Robust Optimization over Decentralized Networks}},
author = {Haomin Bai and Junyan Sun and Quan Cheng and Yuanyu Wan and Lijun Zhang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.29261},
eprint = {2610.29261},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.29261},
}
RIS
TY - UNPB
TI - Group Distributionally Robust Optimization over Decentralized Networks
AU - Haomin Bai
AU - Junyan Sun
AU - Quan Cheng
AU - Yuanyu Wan
AU - Lijun Zhang
PY - 2026
DA - 2026/10/21
PB - arXiv
DO - 10.48550/arxiv.2610.29261
UR - https://arxiv.org/abs/2610.29261
AB - We study group distributionally robust optimization (GDRO) over decentralized networks, where workers collaborate to learn robust models by minimizing the maximum local risk. Existing methods optimize only dual-regularized surrogates and establish guarantees for a globally averaged model that is not available to workers. In this paper, we directly optimize the original GDRO objective and establish guarantees for local models in semi-decentralized (SemiDec) and fully decentralized (Dec) settings. A key difficulty is that applying the primal-dual decomposition from centralized GDRO yields worker-dependent losses whose evaluation requires unavailable cross-worker information. To address this challenge, we develop a novel error decomposition that recasts decentralized GDRO as (i) a decentralized online convex optimization (D-OCO) problem and a prediction-with-expert-advice problem in SemiDec, and as (ii) two coupled D-OCO problems in Dec, with discrepancy terms capturing inconsistencies among local decisions. By isolating consensus errors in analysis, we develop modular frameworks that support different consensus routines. We then establish the \emph{first} per-worker convergence guarantees for optimizing the original GDRO objective over decentralized networks. Instantiations with accelerated gossip and Choco-gossip achieve an O(√((log m)/T)) convergence rate over T update rounds, while experiments demonstrate improved worst-worker performance over baselines.
ER -