Weighted Extensions of the Kolmogorov-Smirnov, Cramer-von Mises, and Anderson-Darling Tests for Assessing Covariate Balance
Abstract
Assessing covariate balance is central to causal inference, but commonly used summary measures may miss distributional differences. I develop weighted extensions of the Kolmogorov-Smirnov (KS), Anderson-Darling (AD), and Cramer-von Mises (CVM) statistics and investigate their calibration when weights are externally supplied or estimated from treatment assignment. With exogenous weights, fixed-weight permutation produced approximately nominal Type I error. With estimated propensity-score weights, fixed-weight permutation was generally conservative, whereas re-estimating weights after unrestricted permutation could be substantially anticonservative when observed and resampled weighting regimes differed. Assignment-mechanism resampling was generally conservative under correct propensity-model specification but could become anticonservative under misspecification. Among the approaches examined for estimated propensity-score weights, fixed-weight permutation therefore provided the most reliable calibration, although it was generally conservative. AD retained strong sensitivity to tail discrepancies, while AD and CVM outperformed KS for diffuse discrepancies. For central discrepancies, KS was most powerful with exogenous weights, whereas AD was most powerful under estimated weighting. An inverse-probability-weighting example illustrates how these tests complement standardized mean differences and variance ratios. Weighted EDF tests provide complementary whole-distribution balance diagnostics. The methods are implemented in the Stata commands kstest, adtest, and cvmtest.
Related
- Weighted derivative estimates for Kolmogorov equations
- Beyond the Mean: A Weighted k-Sample Omnibus Variance-Ratio Statistic for Covariate Balance Diagnostics
- Optimal Covariate Adjustment beyond the Average Treatment Effect: Treated-Population and Overlap-Weighted Estimands
- Caffarelli-Kohn-Nirenberg and Weighted Gaussian Poincaré Inequalities: a complete characterization of sharp L2 stability and Lp extensions
- Weighted Kolmogorov equations: transfer estimates, local boundedness, and Harnack inequalities
BibTeX
@misc{linden2026weighted,
title = {{Weighted Extensions of the Kolmogorov-Smirnov, Cramer-von Mises, and Anderson-Darling Tests for Assessing Covariate Balance}},
author = {Ariel Linden},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2607.21782},
eprint = {2607.21782},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.21782},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Weighted Extensions of the Kolmogorov-Smirnov, Cramer-von Mises, and Anderson-Darling Tests for Assessing Covariate Balance
AU - Ariel Linden
PY - 2026
DA - 2026/10/23
PB - arXiv
DO - 10.48550/arxiv.2607.21782
UR - https://arxiv.org/abs/2607.21782
AB - Assessing covariate balance is central to causal inference, but commonly used summary measures may miss distributional differences. I develop weighted extensions of the Kolmogorov-Smirnov (KS), Anderson-Darling (AD), and Cramer-von Mises (CVM) statistics and investigate their calibration when weights are externally supplied or estimated from treatment assignment. With exogenous weights, fixed-weight permutation produced approximately nominal Type I error. With estimated propensity-score weights, fixed-weight permutation was generally conservative, whereas re-estimating weights after unrestricted permutation could be substantially anticonservative when observed and resampled weighting regimes differed. Assignment-mechanism resampling was generally conservative under correct propensity-model specification but could become anticonservative under misspecification. Among the approaches examined for estimated propensity-score weights, fixed-weight permutation therefore provided the most reliable calibration, although it was generally conservative. AD retained strong sensitivity to tail discrepancies, while AD and CVM outperformed KS for diffuse discrepancies. For central discrepancies, KS was most powerful with exogenous weights, whereas AD was most powerful under estimated weighting. An inverse-probability-weighting example illustrates how these tests complement standardized mean differences and variance ratios. Weighted EDF tests provide complementary whole-distribution balance diagnostics. The methods are implemented in the Stata commands kstest, adtest, and cvmtest.
ER -