Consistency of variational approximations under bounded Kullback-Leibler divergence
Abstract
Variational methods are widely used to approximate posterior distributions in Bayesian inference when exact computation is infeasible. We study when such approximations inherit posterior consistency. Our first result shows that, on a general metric space, a uniform bound on the Kullback-Leibler divergence from the approximating measures to a tight sequence of target measures forces the approximating sequence to be tight. It follows that if the target posteriors converge weakly to a Dirac mass at the true parameter, then any variational sequence with bounded Kullback-Leibler divergence to the targets is also consistent. We also give simple logarithmic-moment conditions that verify this boundedness condition, and illustrate them for smooth generalised posterior distributions.
Related
- Beyond the Kullback--Leibler divergence: the likelihood-ratio distribution as a detectability measure
- Monte Carlo Approximations of Time-Nonlocal Diffusions in Bounded Domains
- Reference-free logged energy-oracle recovery for neural approximations of symmetric coercive variational problems: conforming Riesz reconstruction and archive-level selection
- Geodesic Divergence on Riemannian Planes with Bounded Geometry
- Approximation in norm and energy with bounded functions for autonomous variational problems
BibTeX
@misc{duy2026consistency,
title = {{Consistency of variational approximations under bounded Kullback-Leibler divergence}},
author = {Nguyen Hien Duy and Westerhout Jacob and Guilmeau Thomas and Julyan Arbel},
howpublished = {HAL},
year = {2026},
month = jun,
url = {https://hal.science/hal-05746916v1},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Consistency of variational approximations under bounded Kullback-Leibler divergence
AU - Nguyen Hien Duy
AU - Westerhout Jacob
AU - Guilmeau Thomas
AU - Julyan Arbel
PY - 2026
DA - 2026/06/11
PB - HAL
UR - https://hal.science/hal-05746916v1
AB - Variational methods are widely used to approximate posterior distributions in Bayesian inference when exact computation is infeasible. We study when such approximations inherit posterior consistency. Our first result shows that, on a general metric space, a uniform bound on the Kullback-Leibler divergence from the approximating measures to a tight sequence of target measures forces the approximating sequence to be tight. It follows that if the target posteriors converge weakly to a Dirac mass at the true parameter, then any variational sequence with bounded Kullback-Leibler divergence to the targets is also consistent. We also give simple logarithmic-moment conditions that verify this boundedness condition, and illustrate them for smooth generalised posterior distributions.
ER -