Measuring Differences between Conditional Distributions using Kernel Embeddings
Abstract
Comparing conditional distributions is a fundamental challenge in statistics and machine learning, with applications across a wide range of domains. While proposed methods for measuring discrepancies using kernel embeddings of distributions in a reproducing kernel Hilbert space (RKHS) provide powerful non-parametric techniques, the existing literature remains fragmented and lacks a unified theoretical treatment. This paper addresses this gap by establishing a coherent framework for studying kernel-based methods to measure divergence between conditional distributions through what we refer to as conditional maximum mean discrepancy (CMMD). The CMMD consists of a family of metrics which we call levels, with three special cases each using a different type of RKHS embedding: CMMD0 (conditional mean operators), CMMD1 (conditional mean embeddings), and CMMD2 (joint mean embeddings). We additionally introduce a general level s CMMD, clarifying the required assumptions, and establishing mathematical connections between the levels through the lens of operator-based smoothing. In addition to reviewing previously proposed estimators, we introduce a novel doubly robust estimator for the CMMD that maintains consistency provided the product of errors of the underlying models reduces sufficiently fast. We provide numerical experiments demonstrating that the CMMD effectively captures complex conditional dependencies for statistical testing.
Related
- A General Kernel Framework for Non-CND Distance Measures Using |D|-Dimensional Sparse Landmark Embeddings
- Identified strain kernel A(θ,ω) and a conditional regularity criterion for 3D Navier–Stokes (κ = 3/2; working notes)
- Bilevel optimization for data-driven learning of Koopman embeddings using kernel-based autoencoders
- SOL: Measuring Gaps between Text Distributions by Double Sliced Wasserstein Metrics
- A Kernel-Based Test for Multi-Way Conditional Independence
BibTeX
@misc{moskvichev2026measuring,
title = {{Measuring Differences between Conditional Distributions using Kernel Embeddings}},
author = {Peter Moskvichev and Siu Lun Chau and Dino Sejdinovic},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.02260},
eprint = {2605.02260},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.02260},
}
RIS
TY - UNPB
TI - Measuring Differences between Conditional Distributions using Kernel Embeddings
AU - Peter Moskvichev
AU - Siu Lun Chau
AU - Dino Sejdinovic
PY - 2026
DA - 2026/10/16
PB - arXiv
DO - 10.48550/arxiv.2605.02260
UR - https://arxiv.org/abs/2605.02260
AB - Comparing conditional distributions is a fundamental challenge in statistics and machine learning, with applications across a wide range of domains. While proposed methods for measuring discrepancies using kernel embeddings of distributions in a reproducing kernel Hilbert space (RKHS) provide powerful non-parametric techniques, the existing literature remains fragmented and lacks a unified theoretical treatment. This paper addresses this gap by establishing a coherent framework for studying kernel-based methods to measure divergence between conditional distributions through what we refer to as conditional maximum mean discrepancy (CMMD). The CMMD consists of a family of metrics which we call levels, with three special cases each using a different type of RKHS embedding: CMMD_0 (conditional mean operators), CMMD_1 (conditional mean embeddings), and CMMD_2 (joint mean embeddings). We additionally introduce a general level s CMMD, clarifying the required assumptions, and establishing mathematical connections between the levels through the lens of operator-based smoothing. In addition to reviewing previously proposed estimators, we introduce a novel doubly robust estimator for the CMMD that maintains consistency provided the product of errors of the underlying models reduces sufficiently fast. We provide numerical experiments demonstrating that the CMMD effectively captures complex conditional dependencies for statistical testing.
ER -