Bayesian Cluster-Adjusted Standardised Mean Differences: A Simulation Study
Abstract
Reporting standardised mean differences of continuous psychometric measures is omnipresent in psychology. In a randomised controlled trial with a multilevel structure, one can define a cluster-adjusted version of these quantities naturally taking into account heterogeneous residual variances and accounting for the multilevel structure. In this article, we carefully define the underlying location-scale distributional mixed-effects models and derive cluster-adjusted standardised mean differences using partial pooling. Estimating these quantities is best handled using Bayesian statistics, and we perform an extensive Monte Carlo simulation study to test the performance of these metrics under different simulation configurations. We conclude that in order to measure the a-priori heterogeneous treatment effect across clusters in practically relevant scenarios, it is enough to consider random effects only on the location model and not on the scale model. Even under artificial severe heteroscedasticity, this reduced model performs well both in terms of parameter bias and predictive accuracy. The resulting cluster-adjusted standardised mean difference is an average value across clusters, where the heterogeneity is summarised via its standard deviation.
Related
- Bayesian Estimation under a Censoring-Adjusted Loss Function for Theakash Distribution with Progressive First-Failure Censoring
- Testing Group Differences in Network Models using Bayesian Model Selection: A Tutorial
- Bayesian inference of risk differences for multi-group bilateral data
- Calibrated Bayesian inference for overall and subgroup-specific causal effects in stepped-wedge cluster-randomized trials
- Deterministic Leave-One-Cluster-Out Cross-Validation for Multilevel Bayesian Structural Equation Models
BibTeX
@misc{bahni2026bayesian,
title = {{Bayesian Cluster-Adjusted Standardised Mean Differences: A Simulation Study}},
author = {Yannis Bähni},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.04812},
eprint = {2611.04812},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.04812},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Bayesian Cluster-Adjusted Standardised Mean Differences: A Simulation Study
AU - Yannis Bähni
PY - 2026
DA - 2026/11/03
PB - arXiv
DO - 10.48550/arxiv.2611.04812
UR - https://arxiv.org/abs/2611.04812
AB - Reporting standardised mean differences of continuous psychometric measures is omnipresent in psychology. In a randomised controlled trial with a multilevel structure, one can define a cluster-adjusted version of these quantities naturally taking into account heterogeneous residual variances and accounting for the multilevel structure. In this article, we carefully define the underlying location-scale distributional mixed-effects models and derive cluster-adjusted standardised mean differences using partial pooling. Estimating these quantities is best handled using Bayesian statistics, and we perform an extensive Monte Carlo simulation study to test the performance of these metrics under different simulation configurations. We conclude that in order to measure the a-priori heterogeneous treatment effect across clusters in practically relevant scenarios, it is enough to consider random effects only on the location model and not on the scale model. Even under artificial severe heteroscedasticity, this reduced model performs well both in terms of parameter bias and predictive accuracy. The resulting cluster-adjusted standardised mean difference is an average value across clusters, where the heterogeneity is summarised via its standard deviation.
ER -