What Information Buys in Active Inference
Abstract
Information-theoretic exploration is of primary interest in methodological studies of reinforcement learning (RL) and computational cognitive neuroscience, while the established RL theory builds regret guarantees on confidence sets. This situation leaves open the conditions under which the information-driven exploration found in nature is more advantageous than the exploration of the established RL theory. We put the state-of-the-art methods of these two paradigms, Active Inference (AIF) and Bayes-UCBVI, on an equal footing: both act through the same state abstraction of a tabular MDP with S states, and their greedy forms differ only in the exploration term. We find that AIF's information gain decays too fast to serve as a confidence width, locking greedy agents that explore by it out of the optimal action. Calibrating the information gain by a square root yields calibrated AIF, which attains regret of order √T for an exact abstraction, but no better guarantee than Bayes-UCBVI for a fixed reward. When an adversary chooses rewards from the agent's data, Bayes-UCBVI's value-specific widths lock out, whereas calibrated AIF pays a factor √S, which no goal-memoryless agent avoids on some MDP whose start states the adversary also chooses. Calibrated information gain thus acts as a reward-agnostic ℓ1 confidence set, preferable to value-specific widths when rewards are chosen from the agent's data, not when they are fixed.
Related
- Predicting silent failures of learned inference under restricted access: what measurement physics and prior information explain
- Physics of Information Geometry - Part II: Small-Step Active Inference on the Probability Simplex
- What Does a Benford Test Actually Test? Marginal Conformity, Sampling Structure, and Forensic Inference
- Active Inference as a Convex Markov Decision Process
- Compressed Active Subspaces for Scalable Bayesian Inference
BibTeX
@misc{kandemir2026what,
title = {{What Information Buys in Active Inference}},
author = {Melih Kandemir and Mustafa Mert Çelikok},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.19046},
eprint = {2610.19046},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.19046},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - What Information Buys in Active Inference
AU - Melih Kandemir
AU - Mustafa Mert Çelikok
PY - 2026
DA - 2026/10/13
PB - arXiv
DO - 10.48550/arxiv.2610.19046
UR - https://arxiv.org/abs/2610.19046
AB - Information-theoretic exploration is of primary interest in methodological studies of reinforcement learning (RL) and computational cognitive neuroscience, while the established RL theory builds regret guarantees on confidence sets. This situation leaves open the conditions under which the information-driven exploration found in nature is more advantageous than the exploration of the established RL theory. We put the state-of-the-art methods of these two paradigms, Active Inference (AIF) and Bayes-UCBVI, on an equal footing: both act through the same state abstraction of a tabular MDP with S states, and their greedy forms differ only in the exploration term. We find that AIF's information gain decays too fast to serve as a confidence width, locking greedy agents that explore by it out of the optimal action. Calibrating the information gain by a square root yields calibrated AIF, which attains regret of order √T for an exact abstraction, but no better guarantee than Bayes-UCBVI for a fixed reward. When an adversary chooses rewards from the agent's data, Bayes-UCBVI's value-specific widths lock out, whereas calibrated AIF pays a factor √S, which no goal-memoryless agent avoids on some MDP whose start states the adversary also chooses. Calibrated information gain thus acts as a reward-agnostic ℓ_1 confidence set, preferable to value-specific widths when rewards are chosen from the agent's data, not when they are fixed.
ER -