Personalized Federated Learning through Global Knowledge Distillation and Local Head Adaptation
Abstract
Statistical heterogeneity limits federated learning when a single global classifier cannot represent client-specific label distributions. In this work, we propose Personalized Federated Knowledge Distillation with Head Adaptation (pFedKDH), which aggregates only the shared backbone, keeps persistent client-specific heads, and uses a recalibrated global head as a teacher during local training. Across MNIST, Fashion-MNIST, CIFAR10, and CIFAR100 under class-wise Dirichlet partitions, pFedKDH obtains the best accuracy in most settings, with accuracy gaps up to 37.67\% over the weakest baseline and consistently low standard deviation across repetitions. Component-wise diagnostics and convergence results support the role of persistent heads and distillation-guided local optimization under label-skewed data.
Related
- Geographically Regularized AUC-Maximizing Personalized Federated Learning
- Robust and Sparse Personalized Federated Learning via Gamma-Divergence
- Personalized and differentially private federated learning for mobility data mining via the TrajectoryGuard framework
- Federated Language Models Under Bandwidth Budgets: Distillation Rates and Conformal Coverage
- Personalized Federated Vector Autoregression with Personalization Diversity
BibTeX
@misc{neto2026personalized,
title = {{Personalized Federated Learning through Global Knowledge Distillation and Local Head Adaptation}},
author = {Polycarpo Souza Neto and José Mairton Barros da Silva Júnior and Charles Casimiro Cavalcante},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.17284},
eprint = {2609.17284},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.17284},
}
RIS
TY - UNPB
TI - Personalized Federated Learning through Global Knowledge Distillation and Local Head Adaptation
AU - Polycarpo Souza Neto
AU - José Mairton Barros da Silva Júnior
AU - Charles Casimiro Cavalcante
PY - 2026
DA - 2026/09/15
PB - arXiv
DO - 10.48550/arxiv.2609.17284
UR - https://arxiv.org/abs/2609.17284
AB - Statistical heterogeneity limits federated learning when a single global classifier cannot represent client-specific label distributions. In this work, we propose Personalized Federated Knowledge Distillation with Head Adaptation (pFedKDH), which aggregates only the shared backbone, keeps persistent client-specific heads, and uses a recalibrated global head as a teacher during local training. Across MNIST, Fashion-MNIST, CIFAR10, and CIFAR100 under class-wise Dirichlet partitions, pFedKDH obtains the best accuracy in most settings, with accuracy gaps up to 37.67\% over the weakest baseline and consistently low standard deviation across repetitions. Component-wise diagnostics and convergence results support the role of persistent heads and distillation-guided local optimization under label-skewed data.
ER -