Not Just Why, but When and for Whom: Revisiting Common Estimands in Causal Inference
Abstract
Causal inference commonly defines an individual treatment effect as a quantity that varies across units. But we know such effects, even for a fixed unit, can also vary over time, across repeated studies, or over different methods of measurement, and they can even exhibit inherent stochasticity. In this work, we consider how such additional sources of variation create a need for more general notions of average treatment effects, including some estimands that have yet to be more formally explored by the causal inference community. This perspective also motivates a reconceptualization of our most common existing estimands, such as conditional average, sample average, and individual treatment effects (CATEs, SATEs, and ITEs), as subject to more axes of variation than are typically considered. We explore two main consequences of this reframing in the case where we have data from a single randomized controlled trial: (1) decision making in fixed samples and (2) uncertainty quantification during estimation. We find that different estimands, even for the same unit or the same sample, can qualitatively disagree on whether an effect is positive or negative. We demonstrate these issues empirically through a case study with real data and characterize through a simulation study how our assumptions about these sources of variation, even if we cannot observe the variation, can be consequential for estimation and proper coverage performance. Importantly, these phenomena arise even in the simplest of cases, where all structural and parametric assumptions are satisfied. More broadly, this paper aims to provide a preliminary investigation into an underexplored class of assumptions --- assumptions about unobserved non-confounders --- that should be considered when estimating treatment effects.
Related
- Identification and Estimation of Causal Estimands with Missing Not at Random Data
- Inference for Standard Trial Estimands after Data-Driven Subgroup Discovery
- Adaptive inference for functionals of M-estimands
- Anytime-Valid Inference in Linear Models with Applications to Regression-Adjusted Causal Inference
- Spatial causal inference in the presence of unmeasured confounding and interference
BibTeX
@misc{bynum2026just,
title = {{Not Just Why, but When and for Whom: Revisiting Common Estimands in Causal Inference}},
author = {Lucius E. J. Bynum and George Perrett and Marc A. Scott and Jennifer L. Hill},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.36084},
eprint = {2610.36084},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.36084},
}
RIS
TY - UNPB
TI - Not Just Why, but When and for Whom: Revisiting Common Estimands in Causal Inference
AU - Lucius E. J. Bynum
AU - George Perrett
AU - Marc A. Scott
AU - Jennifer L. Hill
PY - 2026
DA - 2026/10/26
PB - arXiv
DO - 10.48550/arxiv.2610.36084
UR - https://arxiv.org/abs/2610.36084
AB - Causal inference commonly defines an individual treatment effect as a quantity that varies across units. But we know such effects, even for a fixed unit, can also vary over time, across repeated studies, or over different methods of measurement, and they can even exhibit inherent stochasticity. In this work, we consider how such additional sources of variation create a need for more general notions of average treatment effects, including some estimands that have yet to be more formally explored by the causal inference community. This perspective also motivates a reconceptualization of our most common existing estimands, such as conditional average, sample average, and individual treatment effects (CATEs, SATEs, and ITEs), as subject to more axes of variation than are typically considered. We explore two main consequences of this reframing in the case where we have data from a single randomized controlled trial: (1) decision making in fixed samples and (2) uncertainty quantification during estimation. We find that different estimands, even for the same unit or the same sample, can qualitatively disagree on whether an effect is positive or negative. We demonstrate these issues empirically through a case study with real data and characterize through a simulation study how our assumptions about these sources of variation, even if we cannot observe the variation, can be consequential for estimation and proper coverage performance. Importantly, these phenomena arise even in the simplest of cases, where all structural and parametric assumptions are satisfied. More broadly, this paper aims to provide a preliminary investigation into an underexplored class of assumptions --- assumptions about unobserved non-confounders --- that should be considered when estimating treatment effects.
ER -