The Causal Description Gap: Information-Theoretic Separations Across Pearl's Hierarchy
Abstract
Pearl's causal hierarchy shows that observational, interventional, and counterfactual queries are qualitatively distinct. We ask a quantitative version of this question: how many additional bits are needed to specify higher-rung causal answers once lower-rung answers are known? We formalize this via query-class description length, the Kolmogorov complexity of the answers a structural causal model (SCM) gives to a class of queries, with system size given. Our main construction gives acyclic SCMs on n binary variables whose common observational distribution has constant description length, while single-variable intervention answers can require Θ(n2) additional bits. For SCMs with independent noises, fixed finite gate-schema and noise libraries, and at most d parents per variable, a degree-sensitive upper bound gives an observational-interventional gap of at most O(ndlog(en/d)+nlog n). This makes the quadratic construction order-optimal in the dense regime and a rooted-tree construction order-optimal for bounded indegree. The quadratic separation persists under ε-accurate total-variation descriptions for every fixed ε<1/4. At the next rung, even knowing every hard-intervention distribution can leave a Θ(n) counterfactual description gap; a masked-parity construction extends this to Θ(n2). These separation rates hold in the worst case and with high probability under uniform hidden parameters. A general ambiguity-to-bits theorem and Shannon analogue, together with explicit encodings, show that the gaps in our constructions equal the logarithm of residual higher-rung ambiguity up to lower-order terms.
Related
- The Weyl-orbit embedding and a representation-theoretic characterization of the Clifford hierarchy
- Measuring Learned Equivariance: An Information Theoretic Approach
- A Sample-Based Approach for Hierarchical Information-Theoretic Compression of Probabilistic Occupancy Grids
- Conditional Fisher-Information Central Limit Theorems under Log-Concavity with Information-Theoretic Consequences
- Learning Rules, Memorizing Facts: An Information-Theoretic View of Hallucination
BibTeX
@misc{emadi2026causal,
title = {{The Causal Description Gap: Information-Theoretic Separations Across Pearl's Hierarchy}},
author = {Seyed Morteza Emadi},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.02177},
eprint = {2605.02177},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.02177},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - The Causal Description Gap: Information-Theoretic Separations Across Pearl's Hierarchy
AU - Seyed Morteza Emadi
PY - 2026
DA - 2026/10/17
PB - arXiv
DO - 10.48550/arxiv.2605.02177
UR - https://arxiv.org/abs/2605.02177
AB - Pearl's causal hierarchy shows that observational, interventional, and counterfactual queries are qualitatively distinct. We ask a quantitative version of this question: how many additional bits are needed to specify higher-rung causal answers once lower-rung answers are known? We formalize this via query-class description length, the Kolmogorov complexity of the answers a structural causal model (SCM) gives to a class of queries, with system size given. Our main construction gives acyclic SCMs on n binary variables whose common observational distribution has constant description length, while single-variable intervention answers can require Θ(n^2) additional bits. For SCMs with independent noises, fixed finite gate-schema and noise libraries, and at most d parents per variable, a degree-sensitive upper bound gives an observational-interventional gap of at most O(ndlog(en/d)+nlog n). This makes the quadratic construction order-optimal in the dense regime and a rooted-tree construction order-optimal for bounded indegree. The quadratic separation persists under ε-accurate total-variation descriptions for every fixed ε<1/4. At the next rung, even knowing every hard-intervention distribution can leave a Θ(n) counterfactual description gap; a masked-parity construction extends this to Θ(n^2). These separation rates hold in the worst case and with high probability under uniform hidden parameters. A general ambiguity-to-bits theorem and Shannon analogue, together with explicit encodings, show that the gaps in our constructions equal the logarithm of residual higher-rung ambiguity up to lower-order terms.
ER -