PTED: A multi-dimensional two-sample test for scientific inference and generative machine learning
Abstract
Two-sample tests are widely applicable in inference and generative modelling, yet users frequently fall back on heuristics and visual inspection due to lack of an accessible test that operates in multiple dimensions. I present Permutation Test using the Energy Distance (PTED), a Python implementation of a powerful two-sample test described in Székely and Rizzo [2004]. The test statistic is the energy distance, a metric on probability distributions that admits a natural sample estimator built from pairwise distances. A permutation test is run on energy distances to produce an exact two-sample test. Because the statistic depends on the data only through pairwise distances, the test applies in high dimensions, on learned feature representations, at large or small or imbalanced sample sizes, and to any data type on which a distance can be defined. An approximation to the formula allows PTED to scale linearly with both number of dimensions and samples while retaining most discriminative power. I highlight a number of instructive sensitivity tests on PTED alongside other multi-dimensional two-sample tests for comparison. PTED is the only method that is sensitive to all tests, though conventional Maximum Mean Discrepancy is identical for all but an over-fitting test.
Versions
Related
- CausticFlow: An Efficient Machine Learning Framework Combining Neural Differential Equations and Normalizing Flows for Binary Microlensing Parameter Inference
- CausticFlow: An Efficient Machine Learning Framework Combining Neural Differential Equations and Normalizing Flows for Binary Microlensing Parameter Inference
- Accuracy of electric current inference assuming three-dimensional hydrostatic equilibrium
- Bayesian Inference of the Tip of the Red Giant Branch: I. One-Dimensional Analysis
- Learning Gaia: A Generative Model Trained on the Gaia Data Release 3 Source Catalog
BibTeX
@misc{stone2026pted,
title = {{PTED: A multi-dimensional two-sample test for scientific inference and generative machine learning}},
author = {Connor Stone},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.38388},
eprint = {2609.38388},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.38388},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - PTED: A multi-dimensional two-sample test for scientific inference and generative machine learning
AU - Connor Stone
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2609.38388
UR - https://arxiv.org/abs/2609.38388
AB - Two-sample tests are widely applicable in inference and generative modelling, yet users frequently fall back on heuristics and visual inspection due to lack of an accessible test that operates in multiple dimensions. I present Permutation Test using the Energy Distance (PTED), a Python implementation of a powerful two-sample test described in Székely and Rizzo [2004]. The test statistic is the energy distance, a metric on probability distributions that admits a natural sample estimator built from pairwise distances. A permutation test is run on energy distances to produce an exact two-sample test. Because the statistic depends on the data only through pairwise distances, the test applies in high dimensions, on learned feature representations, at large or small or imbalanced sample sizes, and to any data type on which a distance can be defined. An approximation to the formula allows PTED to scale linearly with both number of dimensions and samples while retaining most discriminative power. I highlight a number of instructive sensitivity tests on PTED alongside other multi-dimensional two-sample tests for comparison. PTED is the only method that is sensitive to all tests, though conventional Maximum Mean Discrepancy is identical for all but an over-fitting test.
ER -