Characterizing Client-Independent Stationary Bias for SCAFFOLD and FedAvg in Federated Learning
Abstract
Even when all clients optimize the same strongly convex objective, the global model parameters produced by constant-step federated learning may still have a stationary mean that differs from the minimizer. Does increasing the number of clients eliminate this bias, or can some components persist? Answering this question clarifies the limits of client averaging. To this end, we study two federated learning algorithms, \Scaf{} and matched \FedAvg{}, under full participation, a sufficiently smooth objective, and independent bounded additive gradient noise. Analyzing global stationary balance and within-round local fluctuations yields multidimensional expansions with a shared γ/N term and explicit client-independent γ2 coefficients, where γ is the step size and N is the client count. The \Scaf{} coefficient combines the ordinary local-update contribution found in \FedAvg{} with an additional persistent-control contribution. Although zero-sum controls cancel from the direct server increment, they change local second moments, which interact with third-order curvature to shift the stationary mean. For fixed dimension d and local-step count H, a remainder of OH,d(γ2/N+γ3), uniform in N for sufficiently small γ, identifies the second-order coefficients along arbitrary joint small-step, large-client limits after removing the shared leading term. Controlled experiments assess the predicted coefficients. In real-data analyses, finite-tail mean displacement can account for a substantial share of averaged-model parameter error, and a post-hoc comparison shows directional agreement with the derived curvature--noise structure.
Related
- Resilience Beyond Stationary Client Unavailability: Unlocking Efficient and Unbiased Federated Learning
- Projection-Based Federated Learning on Compact Submanifolds with Geometry-Independent Differential Privacy
- Progressive-Resolution Secure Aggregation for Federated Learning
- Personalized Federated Learning through Global Knowledge Distillation and Local Head Adaptation
- Geographically Regularized AUC-Maximizing Personalized Federated Learning
BibTeX
@misc{tang2026characterizing,
title = {{Characterizing Client-Independent Stationary Bias for SCAFFOLD and FedAvg in Federated Learning}},
author = {Yi-Ping Tang and Guan-Ju Peng},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2608.26765},
eprint = {2608.26765},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.26765},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Characterizing Client-Independent Stationary Bias for SCAFFOLD and FedAvg in Federated Learning
AU - Yi-Ping Tang
AU - Guan-Ju Peng
PY - 2026
DA - 2026/10/18
PB - arXiv
DO - 10.48550/arxiv.2608.26765
UR - https://arxiv.org/abs/2608.26765
AB - Even when all clients optimize the same strongly convex objective, the global model parameters produced by constant-step federated learning may still have a stationary mean that differs from the minimizer. Does increasing the number of clients eliminate this bias, or can some components persist? Answering this question clarifies the limits of client averaging. To this end, we study two federated learning algorithms, \Scaf{} and matched \FedAvg{}, under full participation, a sufficiently smooth objective, and independent bounded additive gradient noise. Analyzing global stationary balance and within-round local fluctuations yields multidimensional expansions with a shared γ/N term and explicit client-independent γ^2 coefficients, where γ is the step size and N is the client count. The \Scaf{} coefficient combines the ordinary local-update contribution found in \FedAvg{} with an additional persistent-control contribution. Although zero-sum controls cancel from the direct server increment, they change local second moments, which interact with third-order curvature to shift the stationary mean. For fixed dimension d and local-step count H, a remainder of O_H,d(γ^2/N+γ^3), uniform in N for sufficiently small γ, identifies the second-order coefficients along arbitrary joint small-step, large-client limits after removing the shared leading term. Controlled experiments assess the predicted coefficients. In real-data analyses, finite-tail mean displacement can account for a substantial share of averaged-model parameter error, and a post-hoc comparison shows directional agreement with the derived curvature--noise structure.
ER -