Conditional Mean Independence, Global Sensitivity Analysis, and Variable Screening Using nearest neighbor Graphs
Abstract
Quantifying how much of the variation in a response is explained by its conditional mean is central to many statistical problems. In this paper, we develop a unified framework for this problem, centered on a normalized conditional mean discrepancy that coincides with the classical Sobol' index for univariate responses and its natural trace-based extension for multivariate responses. We propose a simple estimator of this measure based on nearest-neighbor graphs that can be computed in near-linear time. We establish its consistency and derive its rate of convergence. Further, under the null hypothesis of conditional mean independence, a studentized version of the statistic is asymptotically standard normal. This leads to a computationally efficient test for conditional mean independence that attains the correct asymptotic level and is universally consistent, without requiring bootstrap calibration or sample splitting. Building on the same estimator, we develop a model-free sequential variable-screening procedure that selects a sufficient set with high probability and an exponential error bound. We also discuss extensions of the framework to quantifying interaction effects through higher-order Sobol' indices. Simulations and real-data experiments demonstrate strong power, effective variable screening, and substantial computational gains over existing methods.
Related
- Orthogonal Inference for Conditional Z-Estimation without Cross-Fitting: A Distributional Nearest-Neighbor Approach
- A Kernel-Based Test for Multi-Way Conditional Independence
- Conditional independence and graphical models for rankings
- Sequential Conditional Independence Testing with Machine Learning Models
- Causal Discovery beyond Conditional Independence: Identifiability, Algorithms, and Statistical Inference for Directed Acyclic and Cyclic Graphs
BibTeX
@misc{chatterjee2026conditional,
title = {{Conditional Mean Independence, Global Sensitivity Analysis, and Variable Screening Using nearest neighbor Graphs}},
author = {Anirban Chatterjee and Ziang Niu and Bhaswar B. Bhattacharya},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2607.04692},
eprint = {2607.04692},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.04692},
}
RIS
TY - UNPB
TI - Conditional Mean Independence, Global Sensitivity Analysis, and Variable Screening Using nearest neighbor Graphs
AU - Anirban Chatterjee
AU - Ziang Niu
AU - Bhaswar B. Bhattacharya
PY - 2026
DA - 2026/09/15
PB - arXiv
DO - 10.48550/arxiv.2607.04692
UR - https://arxiv.org/abs/2607.04692
AB - Quantifying how much of the variation in a response is explained by its conditional mean is central to many statistical problems. In this paper, we develop a unified framework for this problem, centered on a normalized conditional mean discrepancy that coincides with the classical Sobol' index for univariate responses and its natural trace-based extension for multivariate responses. We propose a simple estimator of this measure based on nearest-neighbor graphs that can be computed in near-linear time. We establish its consistency and derive its rate of convergence. Further, under the null hypothesis of conditional mean independence, a studentized version of the statistic is asymptotically standard normal. This leads to a computationally efficient test for conditional mean independence that attains the correct asymptotic level and is universally consistent, without requiring bootstrap calibration or sample splitting. Building on the same estimator, we develop a model-free sequential variable-screening procedure that selects a sufficient set with high probability and an exponential error bound. We also discuss extensions of the framework to quantifying interaction effects through higher-order Sobol' indices. Simulations and real-data experiments demonstrate strong power, effective variable screening, and substantial computational gains over existing methods.
ER -