D3MI: an efficient and powerful federated imputation method for bias reduction in the analysis of distributed incomplete data by accounting for within-site correlation and between-site heterogeneity.
- 1Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, Philadelphia, PA, USA. [email protected].
- 2McWilliams School of Biomedical Informatics, University of Texas Health Science Center, Houston, TX, USA.
- 3Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, Philadelphia, PA, USA. [email protected].
Abstract
Background
Electronic health records (EHRs) collected from diverse healthcare institutions offer a rich and representative data source for clinical research. Federated learning enables analysis of these distributed data without sharing sensitive patient-level information, preserving privacy. However, missing data remain a major challenge and can introduce substantial bias if not properly addressed. Very few distributed imputation methods currently exist, and they fail to account for two critical aspects of EHR data: correlation within sites and variability across sites. We aim to fill this important methodological gap.Methods
We propose Distributed Mixed Model-based Multiple Imputation (D3MI), a novel federated imputation method designed to reduce bias in distributed EHRs. D3MI integrates the strengths from federated learning techniques, statistical learning methods for correlated data, and multilevel imputation algorithms to explicitly account for both within-site correlation and between-site heterogeneity using site-specific random effects. It operates under the Missing at Random (MAR) assumption. It preserves privacy by avoiding sharing raw data and features communication and computational efficiency.Results
Through extensive simulation studies, we demonstrate that D3MI outperforms SOTA distributed imputation methods in both accuracy and consistency. We further demonstrate the use of D3MI in a real-world EHR case study involving incomplete and clustered data from participating hospitals in the Georgia Coverdell Acute Stroke Registry.Conclusion
By explicitly modeling the complex structure of distributed EHR data, D3MI addresses key limitations of existing approaches. It provides a powerful and efficient solution for handling missing data in distributed and privacy-sensitive settings and enhances the rigor and reproducibility of collaborative clinical research.
Related
- Comparison of Imputation Strategies for Incomplete Electronic Health Data
- Heterogeneity-calibrated Byzantine-robust distributed composite quantile regression
- More Data, Worse Inference: How the Imputation Model Decides an Interaction Effect
- Bayesian multiple imputation for functional data
- One-step Outcome Imputation: An Alternative to Multiple Imputation
BibTeX
@article{lian2026d3mi,
title = {{D3MI: an efficient and powerful federated imputation method for bias reduction in the analysis of distributed incomplete data by accounting for within-site correlation and between-site heterogeneity.}},
author = {Yi Lian and Xiaoqian Jiang and Qi Long},
journal = {BMC medical informatics and decision making},
year = {2026},
month = oct,
doi = {10.1186/s12911-026-03663-2},
url = {https://europepmc.org/article/MED/42844612},
note = {CC BY},
}
RIS
TY - JOUR
TI - D3MI: an efficient and powerful federated imputation method for bias reduction in the analysis of distributed incomplete data by accounting for within-site correlation and between-site heterogeneity.
AU - Yi Lian
AU - Xiaoqian Jiang
AU - Qi Long
PY - 2026
DA - 2026/10/08
JO - BMC medical informatics and decision making
DO - 10.1186/s12911-026-03663-2
UR - https://europepmc.org/article/MED/42844612
AB - Background Electronic health records (EHRs) collected from diverse healthcare institutions offer a rich and representative data source for clinical research. Federated learning enables analysis of these distributed data without sharing sensitive patient-level information, preserving privacy. However, missing data remain a major challenge and can introduce substantial bias if not properly addressed. Very few distributed imputation methods currently exist, and they fail to account for two critical aspects of EHR data: correlation within sites and variability across sites. We aim to fill this important methodological gap.Methods We propose Distributed Mixed Model-based Multiple Imputation (D3MI), a novel federated imputation method designed to reduce bias in distributed EHRs. D3MI integrates the strengths from federated learning techniques, statistical learning methods for correlated data, and multilevel imputation algorithms to explicitly account for both within-site correlation and between-site heterogeneity using site-specific random effects. It operates under the Missing at Random (MAR) assumption. It preserves privacy by avoiding sharing raw data and features communication and computational efficiency.Results Through extensive simulation studies, we demonstrate that D3MI outperforms SOTA distributed imputation methods in both accuracy and consistency. We further demonstrate the use of D3MI in a real-world EHR case study involving incomplete and clustered data from participating hospitals in the Georgia Coverdell Acute Stroke Registry.Conclusion By explicitly modeling the complex structure of distributed EHR data, D3MI addresses key limitations of existing approaches. It provides a powerful and efficient solution for handling missing data in distributed and privacy-sensitive settings and enhances the rigor and reproducibility of collaborative clinical research.
SN - 1472-6947
ER -