The Earth Moves, But So Does the Bias: Systematic Upward Bias of the Wasserstein (Earth Mover's) Distance and Permutation-Based Null Calibration
Abstract
The Earth Mover's Distance (EMD) is gaining increasing interest among political scientists for assessing similarity in preference distributions. Existing applications predominantly involve relatively large samples and low-dimensional response spaces, which can mitigate finite-sample distortions. However, empirical EMD remains biased upward due to sampling variation in empirical probability measures. This problem is often overlooked and becomes especially severe in high-dimensional or sparse settings, including conjoint distributions that serve as an illustrative example in this paper. As political scientists are broadening their use cases of EMD, this paper cautions against interpreting standard bootstrap uncertainty bounds as a correction for the upward bias of empirical EMD. It proposes a permutation-based null calibration framework for more robust hypothesis testing. As a non-parametric approach, it frees researchers from making directional or distributional shape assumptions. While alternative estimators require these rigid assumptions to correct for upward bias, political science data often fail to meet them in practice. Through four sets of Monte Carlo simulations, this paper demonstrates the utility of this framework. The proposed approach also applies more generally to empirical comparisons of two probability distributions defined on a common metric space, provided that the ground distance between support points is substantively meaningful.
Related
- Bayesian calibration with model discrepancy of a two-group interfacial area transport model for upward bubbly flow in a square duct
- The p-Bures-Wasserstein distance via 3×3 block matrices (with application to p-Bures-Wasserstein barycenters)
- Relations between different definitions of the quantum Wasserstein distance
- λ-geodesic convexity of the squared sliced-Wasserstein distance
- Fast rigid rotational alignment of voxelized 3-D density maps in sliced Wasserstein distance
BibTeX
@misc{hung2026earth,
title = {{The Earth Moves, But So Does the Bias: Systematic Upward Bias of the Wasserstein (Earth Mover's) Distance and Permutation-Based Null Calibration}},
author = {Ho Ting Hung},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2608.09863},
eprint = {2608.09863},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.09863},
}
RIS
TY - UNPB
TI - The Earth Moves, But So Does the Bias: Systematic Upward Bias of the Wasserstein (Earth Mover's) Distance and Permutation-Based Null Calibration
AU - Ho Ting Hung
PY - 2026
DA - 2026/11/05
PB - arXiv
DO - 10.48550/arxiv.2608.09863
UR - https://arxiv.org/abs/2608.09863
AB - The Earth Mover's Distance (EMD) is gaining increasing interest among political scientists for assessing similarity in preference distributions. Existing applications predominantly involve relatively large samples and low-dimensional response spaces, which can mitigate finite-sample distortions. However, empirical EMD remains biased upward due to sampling variation in empirical probability measures. This problem is often overlooked and becomes especially severe in high-dimensional or sparse settings, including conjoint distributions that serve as an illustrative example in this paper. As political scientists are broadening their use cases of EMD, this paper cautions against interpreting standard bootstrap uncertainty bounds as a correction for the upward bias of empirical EMD. It proposes a permutation-based null calibration framework for more robust hypothesis testing. As a non-parametric approach, it frees researchers from making directional or distributional shape assumptions. While alternative estimators require these rigid assumptions to correct for upward bias, political science data often fail to meet them in practice. Through four sets of Monte Carlo simulations, this paper demonstrates the utility of this framework. The proposed approach also applies more generally to empirical comparisons of two probability distributions defined on a common metric space, provided that the ground distance between support points is substantively meaningful.
ER -