How Many Sites, Participants, and Trials? A Tutorial on Multilevel Power Analysis for Multisite Studies with Repeated Measures
Abstract
Multisite collaborations are increasingly common in psychological science. Planning a multisite study with repeated measures involves more than deciding how many participants to recruit. Researchers must also determine how many sites to include and how many observations to collect from each participant. These decisions become particularly challenging when the experimental effect may vary across both participants and sites. Unfortunately, most widely used power analysis tools do not accommodate the complexity of these structures. In this tutorial, we provide a practical introduction to multilevel power analysis for three-level multisite studies. We first discuss how to identify and model the effect of interest, accounting for different sources of heterogeneity and the population to which researchers intend to generalize their findings. We then demonstrate how to estimate statistical power through simulation using {simr} and how to optimize the number of trials, participants, and sites using {mlpwr}, considering their relative costs. Finally, we illustrate how to explore alternative assumptions about effect size and between-site heterogeneity, verify the power of the selected architecture, assess estimation precision, and calculate the minimum detectable effect size. The accompanying R code provides an adaptable workflow to help researchers plan multisite studies that balance their inferential goals with their available resources.
Related
- A tutorial on selecting, translating, and reporting variance priors in Bayesian multilevel models.
- Support Vector Machines for Classification, Regression, and Outlier Detection: A Tutorial
- The Geometry of Informativeness: A Bregman-Centroid Diagnostic for Partial Pooling in Multisite Trials
- Duality in Optimization: A Visual Tutorial
- A Hybrid Spatial Statistical Learning Framework for Individualized Probability Estimation: Application to Multisite Autism Neuroimaging
BibTeX
@misc{francomartinez2026many,
title = {{How Many Sites, Participants, and Trials? A Tutorial on Multilevel Power Analysis for Multisite Studies with Repeated Measures}},
author = {Alicia Franco-Martínez and Miguel A. Vadillo and Ricardo Rey-Sáez},
howpublished = {PsyArXiv},
year = {2026},
month = oct,
doi = {10.31234/osf.io/gwhzv_v2},
url = {https://osf.io/preprints/psyarxiv/gwhzv_v2/},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - How Many Sites, Participants, and Trials? A Tutorial on Multilevel Power Analysis for Multisite Studies with Repeated Measures
AU - Alicia Franco-Martínez
AU - Miguel A. Vadillo
AU - Ricardo Rey-Sáez
PY - 2026
DA - 2026/10/19
PB - PsyArXiv
DO - 10.31234/osf.io/gwhzv_v2
UR - https://osf.io/preprints/psyarxiv/gwhzv_v2/
AB - Multisite collaborations are increasingly common in psychological science. Planning a multisite study with repeated measures involves more than deciding how many participants to recruit. Researchers must also determine how many sites to include and how many observations to collect from each participant. These decisions become particularly challenging when the experimental effect may vary across both participants and sites. Unfortunately, most widely used power analysis tools do not accommodate the complexity of these structures. In this tutorial, we provide a practical introduction to multilevel power analysis for three-level multisite studies. We first discuss how to identify and model the effect of interest, accounting for different sources of heterogeneity and the population to which researchers intend to generalize their findings. We then demonstrate how to estimate statistical power through simulation using {simr} and how to optimize the number of trials, participants, and sites using {mlpwr}, considering their relative costs. Finally, we illustrate how to explore alternative assumptions about effect size and between-site heterogeneity, verify the power of the selected architecture, assess estimation precision, and calculate the minimum detectable effect size. The accompanying R code provides an adaptable workflow to help researchers plan multisite studies that balance their inferential goals with their available resources.
ER -