Asking the Crowd the Right Question: Bias-Cancelling Weights for Federated Learning
Abstract
A federated objective is a weighted sum of client risks, and the weights are almost always fixed in advance. We treat them instead as the only instrument of a wisdom-of-crowds mechanism: clients are noisy views of one truth, each seeing it through an independent distortion that is unbiased across the crowd. That the optimal weights are inversely proportional to the clients' error energies is classical; we begin at the question that answer presupposes, which energies belong there and whether a crowd can recover them from itself. Excess risk on the truth is of the exact order of the aggregate bias energy, so no optimizer can repair a bad weight vector; the truth itself is identifiable only up to a linear tilt, so subtracting estimated client biases provably reproduces uniform weighting. The expected per-client second moments, however, are exactly identified from the law of the crowd's disagreement by a well-conditioned linear inversion, a step random-effects meta-analysis cannot take because a source reports once; their realized counterparts are estimable up to an incoherence floor the algorithm can measure. This yields CROWD, which reads the disagreement off the optimization trajectory at no extra cost and matches a Bayesian minimax lower bound in the same constant: per instance as the horizon grows, and unconditionally as the prior becomes diffuse. For arbitrary distortions it stays competitive with the optimal weights, at a ratio governed by a geometric incoherence the algorithm can measure. On real scans split into sites with their own miscalibrated detectors it attains the oracle excess risk; on a companion federation that pulls bias and noise apart, weighting by noise variance is worse than not weighting at all, and CROWD is not.
Related
- Progressive-Resolution Secure Aggregation for Federated Learning
- Personalized Federated Learning through Global Knowledge Distillation and Local Head Adaptation
- Geographically Regularized AUC-Maximizing Personalized Federated Learning
- Learning to Refer: Inference-Time Collaboration in Decentralized Federated Learning
- A Large-Dimensional Analysis of Least-Squares Federated Learning via Generalized Total Variation Minimization
BibTeX
@misc{kuruzov2026asking,
title = {{Asking the Crowd the Right Question: Bias-Cancelling Weights for Federated Learning}},
author = {Ilya Kuruzov and Dmitrii Vishovan and Kirill Novoselov and Yuriy Dorn and Darina Dvinskikh and Alexander Gasnikov},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.04671},
eprint = {2610.04671},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.04671},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Asking the Crowd the Right Question: Bias-Cancelling Weights for Federated Learning
AU - Ilya Kuruzov
AU - Dmitrii Vishovan
AU - Kirill Novoselov
AU - Yuriy Dorn
AU - Darina Dvinskikh
AU - Alexander Gasnikov
PY - 2026
DA - 2026/10/03
PB - arXiv
DO - 10.48550/arxiv.2610.04671
UR - https://arxiv.org/abs/2610.04671
AB - A federated objective is a weighted sum of client risks, and the weights are almost always fixed in advance. We treat them instead as the only instrument of a wisdom-of-crowds mechanism: clients are noisy views of one truth, each seeing it through an independent distortion that is unbiased across the crowd. That the optimal weights are inversely proportional to the clients' error energies is classical; we begin at the question that answer presupposes, which energies belong there and whether a crowd can recover them from itself. Excess risk on the truth is of the exact order of the aggregate bias energy, so no optimizer can repair a bad weight vector; the truth itself is identifiable only up to a linear tilt, so subtracting estimated client biases provably reproduces uniform weighting. The expected per-client second moments, however, are exactly identified from the law of the crowd's disagreement by a well-conditioned linear inversion, a step random-effects meta-analysis cannot take because a source reports once; their realized counterparts are estimable up to an incoherence floor the algorithm can measure. This yields CROWD, which reads the disagreement off the optimization trajectory at no extra cost and matches a Bayesian minimax lower bound in the same constant: per instance as the horizon grows, and unconditionally as the prior becomes diffuse. For arbitrary distortions it stays competitive with the optimal weights, at a ratio governed by a geometric incoherence the algorithm can measure. On real scans split into sites with their own miscalibrated detectors it attains the oracle excess risk; on a companion federation that pulls bias and noise apart, weighting by noise variance is worse than not weighting at all, and CROWD is not.
ER -