Skip to content
DraconiticResearch
Search Sources API
186,631 studies31 sourcesUpdated 54 min ago
Peer-reviewedCC BYAmerican Journal of Political Science

Designing multi‐site studies for external validity: Site selection via synthetic purposive sampling

  • Naoki Egami1,2
  • Diana Da In Lee3,4
4 affiliations
  1. 1Department of Political Science and Statistics and Data Science Center Massachusetts Institute of Technology Cambridge Massachusetts USA
  2. 2Columbia University
  3. 3Center for the Study of Democratic Institutions Vanderbilt University Nashville Tennessee USA
  4. 4Columbia University - Department of Political Science
Open at sourcePDFDOICite

Abstract

Multi‐site/context studies have become popular strategies to address the most common and challenging external validity concerns about contexts. Under such studies, scholars conduct causal studies in each site and evaluate whether findings generalize across sites. Despite the potential, there has been little guidance on the fundamental research design question—how should we select sites for external validity? Existing approaches have challenges: random sampling of sites is often infeasible, while the current practice of purposive sampling is suboptimal without statistical guarantees. We propose synthetic purposive sampling (SPS), which optimally selects diverse sites for external validity. SPS combines ideas of purposive sampling and the synthetic control method—it selects diverse sites such that nonselected sites are well approximated by the weighted average of the selected sites. We illustrate its general applicability using both experimental and observational studies. Overall, this paper offers a new statistical foundation to design multisite studies for external validity.

Related

  1. Designing Experiments for External Decision Makers
    Sahel Iqbal, Tom Rainforth
    arXivPreprintarXiv licenceFull textStatisticsarXiv
  2. Kernel-based External Validity of Experimental Studies
    Federico Matteucci, Gerhard Neumann
    arXivPreprintarXiv licenceFull textStatisticsarXiv
  3. Honest Inference for Synthetic Controls with Imperfect Fit: Time-Split Validity and the Choice of Anchor
    Rok Spruk
    arXivPreprintCC BY 4.0Full textStatisticsarXiv
  4. Addressing threats to validity in large dataset research
    Wonsuk Yoo, John P A Ioannidis
    PLOS MedicinePeer-reviewedCC BYFull textStatisticsEpidemiology & public healthPLOSEurope PMCEurope PMCPMC
  5. Designing Spatial Treatments
    Stefan Faridani, Michael P. Leung
    arXivPreprintarXiv licenceFull textStatisticsarXiv

Cite

BibTeXRIS
BibTeX
@article{egami2026designing,
  title = {{Designing multi‐site studies for external validity: Site selection via synthetic purposive sampling}},
  author = {Naoki Egami and Diana Da In Lee},
  journal = {American Journal of Political Science},
  year = {2026},
  month = sep,
  doi = {10.1111/ajps.70070},
  url = {https://doi.org/10.1111/ajps.70070},
  note = {CC BY},
}
RIS
TY  - JOUR
TI  - Designing multi‐site studies for external validity: Site selection via synthetic purposive sampling
AU  - Naoki Egami
AU  - Diana Da In Lee
PY  - 2026
DA  - 2026/09/07
JO  - American Journal of Political Science
DO  - 10.1111/ajps.70070
UR  - https://doi.org/10.1111/ajps.70070
AB  - Multi‐site/context studies have become popular strategies to address the most common and challenging external validity concerns about contexts. Under such studies, scholars conduct causal studies in each site and evaluate whether findings generalize across sites. Despite the potential, there has been little guidance on the fundamental research design question—how should we select sites for external validity? Existing approaches have challenges: random sampling of sites is often infeasible, while the current practice of purposive sampling is suboptimal without statistical guarantees. We propose synthetic purposive sampling (SPS), which optimally selects diverse sites for external validity. SPS combines ideas of purposive sampling and the synthetic control method—it selects diverse sites such that nonselected sites are well approximated by the weighted average of the selected sites. We illustrate its general applicability using both experimental and observational studies. Overall, this paper offers a new statistical foundation to design multisite studies for external validity.
SN  - 0092-5853
SN  - 1540-5907
ER  -