How Wrong Can a Good Predictor Be? Diverging Updates with Vanishing Predictive KL
Abstract
Accurate posterior prediction need not require accurate approximation of Bayesian updates. We prove that an unbounded gap between the update maps can coexist with vanishing predictive KL for every fixed finite K≥2 in a stationary symmetric Gaussian HMM. Exact Bayesian mixing and an explicit deterministic radial filter act on the same K−1 belief coordinates. As q→0+, their separation in centered logits in the worst case grows at least linearly in the natural confidence scale LK(q), while their categorical DKL(exact|radial) vanishes at the same explicit witness. Along stationary HMM trajectories, the expected terminal KL between filtered posteriors also converges to zero at H(q)=⌈−log(q)/c⌉+1. Typical blocks without switches drive both filters into a common confidence cone, where softmax curvature suppresses their disagreement; a single Gaussian maximal event controls adaptive noise. A sweep with equally spaced Gaussians over K∈{2,4,8} illustrates the opposing trends, and binary controls at long horizons compare saturating and nonsaturating recurrences. The result isolates two missing links between internal update gaps and predictive cost: the contribution of separating states to expected loss and decoder sensitivity. Thus even an unbounded internal update gap does not by itself certify predictive failure. The construction is fixed in K and does not provide a universal criterion for when compression is harmless or characterize when internal gaps must incur task loss.
Related
- Coordinated Lane-Level Variable Speed Limits and Ramp Metering for Successive Weaving Segments Considering Merging/Diverging Risks: A Hybrid Model Predictive Control and Multi-Agent Reinforcement Learning Approach
- Drift-Resilient Predictive Interference Management: Online Adaptation of an Extreme-Value Predictor
- A Diverging History Clock Is Not a Blow-up Proof: Lean-Checked Lemmas Showing Why a Section 10 Navier-Stokes Argument Does Not Certify Fefferman (C) or (D)
- Network inference for SDEs with diverging dimension under small noise
- Lane-Emden equation for the superposition of a local and nonlocal operator with the "wrong sign"
BibTeX
@misc{wen2026wrong,
title = {{How Wrong Can a Good Predictor Be? Diverging Updates with Vanishing Predictive KL}},
author = {Qifu Wen and Shuaijun Liu and Zihan Zhou and Xi Zeng and Ningxin Su},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.11132},
eprint = {2609.11132},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.11132},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - How Wrong Can a Good Predictor Be? Diverging Updates with Vanishing Predictive KL
AU - Qifu Wen
AU - Shuaijun Liu
AU - Zihan Zhou
AU - Xi Zeng
AU - Ningxin Su
PY - 2026
DA - 2026/09/10
PB - arXiv
DO - 10.48550/arxiv.2609.11132
UR - https://arxiv.org/abs/2609.11132
AB - Accurate posterior prediction need not require accurate approximation of Bayesian updates. We prove that an unbounded gap between the update maps can coexist with vanishing predictive KL for every fixed finite K≥2 in a stationary symmetric Gaussian HMM. Exact Bayesian mixing and an explicit deterministic radial filter act on the same K−1 belief coordinates. As q→0^+, their separation in centered logits in the worst case grows at least linearly in the natural confidence scale L_K(q), while their categorical D_KL(exact|radial) vanishes at the same explicit witness. Along stationary HMM trajectories, the expected terminal KL between filtered posteriors also converges to zero at H(q)=⌈−log(q)/c⌉+1. Typical blocks without switches drive both filters into a common confidence cone, where softmax curvature suppresses their disagreement; a single Gaussian maximal event controls adaptive noise. A sweep with equally spaced Gaussians over K∈{2,4,8} illustrates the opposing trends, and binary controls at long horizons compare saturating and nonsaturating recurrences. The result isolates two missing links between internal update gaps and predictive cost: the contribution of separating states to expected loss and decoder sensitivity. Thus even an unbounded internal update gap does not by itself certify predictive failure. The construction is fixed in K and does not provide a universal criterion for when compression is harmless or characterize when internal gaps must incur task loss.
ER -