Comparison of parametric versus machine-learning multiple imputation in clinical trials with missing continuous outcomes.
- 1MRC-Biostatistics Unit, University of Cambridge, East Forvie Building, Forvie Site, Robinson Way, Cambridge Biomedical Campus, Cambridge, CB2 0SR, UK. [email protected].
- 2Department of Medical Statistics, London School of Hygiene and Tropical Medicine, Keppel Street, London, WC1E 7HT, UK.
- 3Institute of Psychiatry, King's College London, 16 De Crespigny Park, London, SE5 8AB, UK.
Abstract
Background
Flexible machine-learning (ML) models to generate imputations within the Multiple Imputation (MI) framework has recently gained traction, particularly in non-randomised observational settings. For randomised controlled trials (RCTs), it is unclear whether ML approaches to MI, in combination with Rubin's Rules, provide valid inference in terms bias, confidence interval coverage, mean squared error (MSE), Type I error and power of the treatment effect estimator.Methods
We conducted two simulation studies in RCT settings that have an incomplete continuous outcome but fully observed covariates and treatment assignment. We compared Complete Cases, standard MI (MI-norm), MI with predictive mean matching (MI-PMM) and ML-based approaches to MI, including classification and regression trees (MI-CART), Random Forests (MI-RF) and SuperLearner when outcomes are missing completely at random or missing at random conditional on treatment/covariate. The first simulation explored a cross-sectional outcome with non-linear covariate-outcome relationships in the presence/absence of covariate-treatment interactions. The second simulation explored skewed repeated measures, motivated by a trial with digital outcomes.Results
For the cross-sectional simulation without interaction, we found that Complete Cases yielded valid inference; MI-norm performed similarly, except when there is a non-linear covariate-outcome relationship and missingness depends on the covariate. ML approaches led to smaller MSE in specific non-linear settings, but provided unreliable inference for others. MI-PMM, which is the default setting in the R package mice, led to unreliable inference under some settings. In the presence of complex treatment-covariate interactions, performing MI separately by arm, either with MI-norm, MI-RF or MI-CART, provided inference that had comparable or better properties compared to Complete Cases when the analysis model omits the interaction term. In the repeated measures setting, ML approaches to MI and MI-PMM led to bias and under- or over-coverage, particularly when missingness depended on treatment.Conclusion
Based on the simulation findings, Complete Cases and MI-norm are more appropriate than ML approaches to MI for making inference, especially for late phase RCTs where Type I error control is crucial. While ML approaches may provide gains in MSE for complex covariate-outcome relationships, results should be interpreted with the caveat that Rubin's Rules are not guaranteed to be valid when used with ML imputation methods, and can lead to bias in the estimated effect and/or its standard error.
Related
- Robust variance estimation for multiple imputation using predictive mean matching or parametric imputation models
- Bayesian multiple imputation for functional data
- One-step Outcome Imputation: An Alternative to Multiple Imputation
- Reference-based multiple imputation for longitudinal bounded discrete outcomes
- Bayesian Multiple Imputation for Functional Data
BibTeX
@article{tackney2026comparison,
title = {{Comparison of parametric versus machine-learning multiple imputation in clinical trials with missing continuous outcomes.}},
author = {Mia S Tackney and Jonathan W Bartlett and Elizabeth Williamson and Kim May Lee},
journal = {BMC medical research methodology},
year = {2026},
month = aug,
doi = {10.1186/s12874-026-02962-6},
url = {https://europepmc.org/article/MED/42958074},
note = {CC BY},
}
RIS
TY - JOUR
TI - Comparison of parametric versus machine-learning multiple imputation in clinical trials with missing continuous outcomes.
AU - Mia S Tackney
AU - Jonathan W Bartlett
AU - Elizabeth Williamson
AU - Kim May Lee
PY - 2026
DA - 2026/08/03
JO - BMC medical research methodology
DO - 10.1186/s12874-026-02962-6
UR - https://europepmc.org/article/MED/42958074
AB - Background Flexible machine-learning (ML) models to generate imputations within the Multiple Imputation (MI) framework has recently gained traction, particularly in non-randomised observational settings. For randomised controlled trials (RCTs), it is unclear whether ML approaches to MI, in combination with Rubin's Rules, provide valid inference in terms bias, confidence interval coverage, mean squared error (MSE), Type I error and power of the treatment effect estimator.Methods We conducted two simulation studies in RCT settings that have an incomplete continuous outcome but fully observed covariates and treatment assignment. We compared Complete Cases, standard MI (MI-norm), MI with predictive mean matching (MI-PMM) and ML-based approaches to MI, including classification and regression trees (MI-CART), Random Forests (MI-RF) and SuperLearner when outcomes are missing completely at random or missing at random conditional on treatment/covariate. The first simulation explored a cross-sectional outcome with non-linear covariate-outcome relationships in the presence/absence of covariate-treatment interactions. The second simulation explored skewed repeated measures, motivated by a trial with digital outcomes.Results For the cross-sectional simulation without interaction, we found that Complete Cases yielded valid inference; MI-norm performed similarly, except when there is a non-linear covariate-outcome relationship and missingness depends on the covariate. ML approaches led to smaller MSE in specific non-linear settings, but provided unreliable inference for others. MI-PMM, which is the default setting in the R package mice, led to unreliable inference under some settings. In the presence of complex treatment-covariate interactions, performing MI separately by arm, either with MI-norm, MI-RF or MI-CART, provided inference that had comparable or better properties compared to Complete Cases when the analysis model omits the interaction term. In the repeated measures setting, ML approaches to MI and MI-PMM led to bias and under- or over-coverage, particularly when missingness depended on treatment.Conclusion Based on the simulation findings, Complete Cases and MI-norm are more appropriate than ML approaches to MI for making inference, especially for late phase RCTs where Type I error control is crucial. While ML approaches may provide gains in MSE for complex covariate-outcome relationships, results should be interpreted with the caveat that Rubin's Rules are not guaranteed to be valid when used with ML imputation methods, and can lead to bias in the estimated effect and/or its standard error.
SN - 1471-2288
ER -