Imputation-Based Harmonization Mitigates Site Effects Without Data Leakage in Machine Learning Studies
- 1University of Pennsylvania
Abstract
Neuroimaging studies that pool data across clinical sites often suffer from site effects --- variability in imaging measures that arises from technical heterogeneity across sites as opposed to true biological signal. While numerous harmonization methods have been proposed to remove site effects from imaging features, less attention has been placed on how to incorporate harmonization models into common machine-learning pipelines. We demonstrate that current approaches for integration either suffer from data leakage, fail to fully remove site effects from the target data, or attenuate true biological associations during the harmonization process. To address these issues, we propose Multiple Imputation for Removing Technical Heterogeneity (MIRTH), which uses imputed outcomes to harmonize the test data. We benchmark MIRTH's performance using both simulated and real-world volumetric data from the Alzheimer's Disease Neuroimaging Initiative and the Baltimore Longitudinal Study of Aging, demonstrating that MIRTH can achieve high predictive accuracy while avoiding inflated performance when the outcome is imbalanced across clinical sites.
Related
- A Context-Aware Framework for Biological Health Index Harmonization and Machine Learning-Aided Health Forecasting of Waterbodies
- Window leakage in clinical EEG machine learning: matched subject-independent evaluation across ADHD and Parkinson cohorts
- Leakage-controlled benchmarking reveals generalization limits of deep learning for protein-ligand binding affinity prediction
- A structural antibody benchmark for leakage-aware deep-learning evaluation
- Towards reliable data augmentation in machine learning: Practices to prevent data leakage
BibTeX
@misc{hillman2026imputationbased,
title = {{Imputation-Based Harmonization Mitigates Site Effects Without Data Leakage in Machine Learning Studies}},
author = {N. Hillman and A. Chen and F. Hu and S. Vandekar and R. Melhem and L. Beason-Held and T. Satterthwaite and C. Davatzikos and H. Shou and R. Shinohara},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.28.755204},
url = {https://www.biorxiv.org/content/10.64898/2026.09.28.755204v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Imputation-Based Harmonization Mitigates Site Effects Without Data Leakage in Machine Learning Studies
AU - N. Hillman
AU - A. Chen
AU - F. Hu
AU - S. Vandekar
AU - R. Melhem
AU - L. Beason-Held
AU - T. Satterthwaite
AU - C. Davatzikos
AU - H. Shou
AU - R. Shinohara
PY - 2026
DA - 2026/09/28
PB - bioRxiv
DO - 10.64898/2026.09.28.755204
UR - https://www.biorxiv.org/content/10.64898/2026.09.28.755204v1
AB - Neuroimaging studies that pool data across clinical sites often suffer from site effects --- variability in imaging measures that arises from technical heterogeneity across sites as opposed to true biological signal. While numerous harmonization methods have been proposed to remove site effects from imaging features, less attention has been placed on how to incorporate harmonization models into common machine-learning pipelines. We demonstrate that current approaches for integration either suffer from data leakage, fail to fully remove site effects from the target data, or attenuate true biological associations during the harmonization process. To address these issues, we propose Multiple Imputation for Removing Technical Heterogeneity (MIRTH), which uses imputed outcomes to harmonize the test data. We benchmark MIRTH's performance using both simulated and real-world volumetric data from the Alzheimer's Disease Neuroimaging Initiative and the Baltimore Longitudinal Study of Aging, demonstrating that MIRTH can achieve high predictive accuracy while avoiding inflated performance when the outcome is imbalanced across clinical sites.
ER -