A Hierarchy of Entropy-Shapley Games for Multivariate Predictive Uncertainty
Abstract
Modern probabilistic machine learning models increasingly produce multivariate outputs with complex dependence structure, from multi-step time-series forecasts to sample path predictions. Understanding which input features drive the predictive uncertainty is important for risk-aware decisions, model diagnostics, and deciding whether the uncertainty should be mitigated or hedged against. This attribution problem requires a choice of how dependencies between output components are treated. Existing approaches reduce the output to a scalar through aggregation or projection before attribution, thereby obscuring whether features affect marginal uncertainty, dependence structure, or both, while component-wise analyses can miss dependence effects entirely. We close this gap by introducing a hierarchy of three entropy-based Shapley games that make this output-side choice explicit for any ordered multivariate outcome, ranging from per-component marginal entropy to fully joint entropy. The hierarchy isolates a cross-component attribution term that captures how each feature shifts the dependence between output components, a quantity invisible to component-wise methods. We establish a chain-rule decomposition of the joint attribution and characterize the cross-component term through conditional total correlation, providing both closed-form and sample-based estimators. Finally, we demonstrate how the framework captures differences in learned joint structure across probabilistic models from distributional regression to a zero-shot time series foundation model.
Related
- Improved estimation of differential entropy of a multivariate Gaussian distribution
- Generating Functions and the Entropy Hierarchy of Strongly Connected Digraphs
- Linear-Time Multivariate Unsupervised Feature Selection Using Nyström-Based Entropy Approximation
- Multivariate conformal uncertainty propagation in multitask atomistic simulation: Successes and pitfalls
- Bagged Martingale Posteriors: Calibrated Uncertainty Quantification for Predictive Resampling
BibTeX
@misc{koenen2026hierarchy,
title = {{A Hierarchy of Entropy-Shapley Games for Multivariate Predictive Uncertainty}},
author = {Niklas Koenen and Claudia Battistin and Jeriek Van den Abeele and Martin Jullum},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.35217},
eprint = {2609.35217},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.35217},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - A Hierarchy of Entropy-Shapley Games for Multivariate Predictive Uncertainty
AU - Niklas Koenen
AU - Claudia Battistin
AU - Jeriek Van den Abeele
AU - Martin Jullum
PY - 2026
DA - 2026/09/28
PB - arXiv
DO - 10.48550/arxiv.2609.35217
UR - https://arxiv.org/abs/2609.35217
AB - Modern probabilistic machine learning models increasingly produce multivariate outputs with complex dependence structure, from multi-step time-series forecasts to sample path predictions. Understanding which input features drive the predictive uncertainty is important for risk-aware decisions, model diagnostics, and deciding whether the uncertainty should be mitigated or hedged against. This attribution problem requires a choice of how dependencies between output components are treated. Existing approaches reduce the output to a scalar through aggregation or projection before attribution, thereby obscuring whether features affect marginal uncertainty, dependence structure, or both, while component-wise analyses can miss dependence effects entirely. We close this gap by introducing a hierarchy of three entropy-based Shapley games that make this output-side choice explicit for any ordered multivariate outcome, ranging from per-component marginal entropy to fully joint entropy. The hierarchy isolates a cross-component attribution term that captures how each feature shifts the dependence between output components, a quantity invisible to component-wise methods. We establish a chain-rule decomposition of the joint attribution and characterize the cross-component term through conditional total correlation, providing both closed-form and sample-based estimators. Finally, we demonstrate how the framework captures differences in learned joint structure across probabilistic models from distributional regression to a zero-shot time series foundation model.
ER -