Neural Information Causality: A Random-Access Capacity Law for Representation Learning
Abstract
How can a learned representation remain useful when the question it must answer is chosen only after encoding? Preserving information is essential, but does not by itself ensure access to the requested target. We address this problem by deriving "neural information causality (Neural-IC)," a representation law that connects causal access to the physically specified interface capacity. For a fresh database of independent bits, the aggregate information recoverable across later queries cannot exceed that capacity, even with pre-established quantum resources. A decomposition separates capacity slack, information organized jointly across targets, and losses in producing answers. Exhaustive evaluation of trained binary codes reveals why nearly full use of a memory's information budget can coexist with limited retrieval. In a nested Bell-correlation construction, quantum assistance improves average retrieval over the optimal classical one-bit benchmark within the same communicated budget. These results give representation learning a design principle that connects information capacity to the organization and extraction of target information. By locating the constraints on later answers, the framework can inform memory allocation, encoding objectives, and decoder outputs, providing a basis for seeking better retrieval within a fixed budget.
Related
- Emergent Generalization by Representation Learning in Artificial Neural Networks
- Information Causality Characterizes the Set of Quantum Correlations in the Simplest Bell Scenario
- Almost-Quantum Correlations Violate Information Causality
- Bounding single-copy information causality violations
- Information causality does not single out the set of quantum correlations
BibTeX
@misc{bang2026neural,
title = {{Neural Information Causality: A Random-Access Capacity Law for Representation Learning}},
author = {Jeongho Bang and Marcin Pawłowski},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.09316},
eprint = {2605.09316},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.09316},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Neural Information Causality: A Random-Access Capacity Law for Representation Learning
AU - Jeongho Bang
AU - Marcin Pawłowski
PY - 2026
DA - 2026/10/24
PB - arXiv
DO - 10.48550/arxiv.2605.09316
UR - https://arxiv.org/abs/2605.09316
AB - How can a learned representation remain useful when the question it must answer is chosen only after encoding? Preserving information is essential, but does not by itself ensure access to the requested target. We address this problem by deriving "neural information causality (Neural-IC)," a representation law that connects causal access to the physically specified interface capacity. For a fresh database of independent bits, the aggregate information recoverable across later queries cannot exceed that capacity, even with pre-established quantum resources. A decomposition separates capacity slack, information organized jointly across targets, and losses in producing answers. Exhaustive evaluation of trained binary codes reveals why nearly full use of a memory's information budget can coexist with limited retrieval. In a nested Bell-correlation construction, quantum assistance improves average retrieval over the optimal classical one-bit benchmark within the same communicated budget. These results give representation learning a design principle that connects information capacity to the organization and extraction of target information. By locating the constraints on later answers, the framework can inform memory allocation, encoding objectives, and decoder outputs, providing a basis for seeking better retrieval within a fixed budget.
ER -