Design-based effect sizes and their synthesis in randomized multiple-baseline experiments
Abstract
Effect sizes for single-case experimental designs usually rely on models for the baseline trend and the serial dependence, which are challenging to check with few sessions. We develop design-based effect sizes for multiple-baseline experiments with a randomized intervention onset. We define case-level and pooled standardized effects with potential outcomes and propose weighted estimators that are unbiased or asymptotically unbiased without an outcome model. The usual difference between phase means is generally biased when the untreated outcome changes over sessions. For research synthesis, we provide variance estimators and asymptotic confidence intervals that require no model for serial dependence, and variance formulas for planning. Simulations and three randomized studies of reading and writing interventions illustrate the methods.
Related
- The promises of multiple experiments: identifying joint distribution of potential outcomes
- Regression adjustment in completely randomized experiments with many covariates
- A Primer on the Analysis of Randomized Experiments and a Survey of some Recent Advances
- Fisher's Ideas and the Design of Field Experiments in Agronomy and Plant Breeding
- Design-Based Confidence Sequences: A General Approach to Risk Mitigation in Panel Experiments
BibTeX
@misc{chen2026designbased,
title = {{Design-based effect sizes and their synthesis in randomized multiple-baseline experiments}},
author = {Xinyuan Chen and Zhigao Liang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.13001},
eprint = {2610.13001},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.13001},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Design-based effect sizes and their synthesis in randomized multiple-baseline experiments
AU - Xinyuan Chen
AU - Zhigao Liang
PY - 2026
DA - 2026/10/09
PB - arXiv
DO - 10.48550/arxiv.2610.13001
UR - https://arxiv.org/abs/2610.13001
AB - Effect sizes for single-case experimental designs usually rely on models for the baseline trend and the serial dependence, which are challenging to check with few sessions. We develop design-based effect sizes for multiple-baseline experiments with a randomized intervention onset. We define case-level and pooled standardized effects with potential outcomes and propose weighted estimators that are unbiased or asymptotically unbiased without an outcome model. The usual difference between phase means is generally biased when the untreated outcome changes over sessions. For research synthesis, we provide variance estimators and asymptotic confidence intervals that require no model for serial dependence, and variance formulas for planning. Simulations and three randomized studies of reading and writing interventions illustrate the methods.
ER -