Redundancy and synergy in multivariate Gaussians via the Blackwell order
Abstract
The goal of the partial information decomposition (PID) is to quantify the redundant and synergistic information that multiple sources provide about a target. PID has many applications in machine learning, neuroscience, and other fields, but defining and computing it for high-dimensional continuous systems remains challenging. Here, we define a PID for multivariate Gaussian systems based on the Blackwell order, which formalizes when one channel is more informative than another. We prove that Gaussian channels are optimal for extracting both redundant and union information, yielding an intuitive geometric interpretation and an efficient numerical algorithm for the PID. Our union information and synergy coincide with the well-known BROJA measures, and we derive closed-form expressions for both in the case of two sources. We also argue that Blackwell redundancy (which differs from BROJA) is the only existing redundancy measure that satisfies a set of natural desiderata. We demonstrate the scalability of our method on systems with up to a thousand dimensions or sources. Our approach is illustrated on an optimal control problem, where it identifies redundant and synergistic interactions between sensor and memory.
Related
- Between Hi-Fi and MMI: the overlap of redundancy and synergy in partial information decomposition
- Accelerating Fourier--Motzkin elimination: redundancy removal and the choice of variable elimination order
- Matrix roots of order n of a multivariate polynomial
- Adaptive Conformal Inference through the Lens of Blackwell Approachability
- Gaussians Do Not Always Maximize Mixed-Norm Strichartz Inequalities for the Schrödinger Equation
BibTeX
@misc{kolchinsky2026redundancy,
title = {{Redundancy and synergy in multivariate Gaussians via the Blackwell order}},
author = {Artemy Kolchinsky},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.07360},
eprint = {2610.07360},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.07360},
note = {CC0},
}
RIS
TY - UNPB
TI - Redundancy and synergy in multivariate Gaussians via the Blackwell order
AU - Artemy Kolchinsky
PY - 2026
DA - 2026/10/05
PB - arXiv
DO - 10.48550/arxiv.2610.07360
UR - https://arxiv.org/abs/2610.07360
AB - The goal of the partial information decomposition (PID) is to quantify the redundant and synergistic information that multiple sources provide about a target. PID has many applications in machine learning, neuroscience, and other fields, but defining and computing it for high-dimensional continuous systems remains challenging. Here, we define a PID for multivariate Gaussian systems based on the Blackwell order, which formalizes when one channel is more informative than another. We prove that Gaussian channels are optimal for extracting both redundant and union information, yielding an intuitive geometric interpretation and an efficient numerical algorithm for the PID. Our union information and synergy coincide with the well-known BROJA measures, and we derive closed-form expressions for both in the case of two sources. We also argue that Blackwell redundancy (which differs from BROJA) is the only existing redundancy measure that satisfies a set of natural desiderata. We demonstrate the scalability of our method on systems with up to a thousand dimensions or sources. Our approach is illustrated on an optimal control problem, where it identifies redundant and synergistic interactions between sensor and memory.
ER -