Efficacy of statistical and design interventions for improving reproducibility in biomedical research
Abstract
Over the past two decades, failures to replicate published findings in biomedical research have led to a perceived reproducibility crisis. Several interventions at the level of inferential statistics and experimental design have been proposed for improving reproducibility. These include raising the power by increasing the sample size, redefining the threshold for statistical significance, focusing on a minimally important difference, adding a penalising term for between-study variation, and combining data from multiple study centres. We used simulations to investigate the effectiveness of these interventions on measures of reproducibility. Artificial data sets were created by repeatedly sampling from defined parametric distributions based on published biomedical studies (i.e. real-world data sets). Reproducibility measures included the probability of positive findings, the probability of consistent inferential outcomes and the chance of capturing the true effect size. The results were mixed: some of the proposed interventions could improve reproducibility, while others had detrimental effects on reproducibility. Notably, the interventions affected the various aspects of reproducibility differently.
Related
- Improving Inference from Reported Concentrations in Environmental Surveillance by Modeling the Statistical Features of Digital PCR
- Reproducibility of social science research using aggregate statistics with noise infused for differential privacy
- Reviewing reproducibility and replicability in 'Deconstructing gastrulation at single-cell resolution' by Stern et al. (2022).
- DICD-IE: an expert-annotated benchmark for evaluating pretrained biomedical language models in dental implant literature and clinical knowledge extraction
- Autonomous biomedical research with an artificial intelligence agent
BibTeX
@article{voelkl2026efficacy,
title = {{Efficacy of statistical and design interventions for improving reproducibility in biomedical research}},
author = {Bernhard Voelkl and Rachel Heyard and Laura Caquelin and Gustav Nilsonne and Kimberley E. Wever and Hanno Würbel},
journal = {BMC Biology},
year = {2026},
month = oct,
doi = {10.1186/s12915-026-02771-9},
url = {https://link.springer.com/article/10.1186/s12915-026-02771-9},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Efficacy of statistical and design interventions for improving reproducibility in biomedical research
AU - Bernhard Voelkl
AU - Rachel Heyard
AU - Laura Caquelin
AU - Gustav Nilsonne
AU - Kimberley E. Wever
AU - Hanno Würbel
PY - 2026
DA - 2026/10/21
JO - BMC Biology
DO - 10.1186/s12915-026-02771-9
UR - https://link.springer.com/article/10.1186/s12915-026-02771-9
AB - Over the past two decades, failures to replicate published findings in biomedical research have led to a perceived reproducibility crisis. Several interventions at the level of inferential statistics and experimental design have been proposed for improving reproducibility. These include raising the power by increasing the sample size, redefining the threshold for statistical significance, focusing on a minimally important difference, adding a penalising term for between-study variation, and combining data from multiple study centres. We used simulations to investigate the effectiveness of these interventions on measures of reproducibility. Artificial data sets were created by repeatedly sampling from defined parametric distributions based on published biomedical studies (i.e. real-world data sets). Reproducibility measures included the probability of positive findings, the probability of consistent inferential outcomes and the chance of capturing the true effect size. The results were mixed: some of the proposed interventions could improve reproducibility, while others had detrimental effects on reproducibility. Notably, the interventions affected the various aspects of reproducibility differently.
SN - 1741-7007
ER -