Rényi-Regularized Flow-Based Variational Inference for Bayesian Calibration
Abstract
Variational inference with normalizing flows provides a flexible posterior approximation for Bayesian model calibration, but struggles when the posterior has several modes separated by regions of low probability mass. We study this behavior in a two-parameter twin experiment and regularize the variational objective. Under Gaussian observation errors and a Gaussian prior, the corresponding minimization objective is the expected Tikhonov calibration objective minus the differential entropy of the approximating distribution. However, the expected Tikhonov term favors concentration near one of the modes, while entropy maximization does not guarantee multimodality. We introduce a regularization term based on Rényi divergence that encourages the approximate posterior to cover the prior support. In the connected bimodal case, a flow approximation represents both modes; in the case with well-separated modes, the flow and Gaussian mixture frequently miss one mode. The introduced regularization reduces the near-collapse fraction from 50% to 25%. While the regularization improves mode coverage in this experiment, it changes the Bayesian variational objective, introduces a trade-off in approximation accuracy, and does not provide a general solution to mode collapse.
Related
- Structure-aware Bayesian full waveform inversion via diffusion-regularized Stein variational gradient descent
- Asymptotically Valid Laplace Variational Inference for Bayesian Nonparametric Clustering of Marked Point Patterns
- Variational Bayesian Inference Based Progressive Multi-Target MIMO Sensing with Nuisance Parameters
- Calibration-Free Robust Bayesian Inference for Discrete Unnormalized Models
- Concentration and Calibration in Predictive Bayesian Inference
BibTeX
@misc{ngarti2026rényiregularized,
title = {{Rényi-Regularized Flow-Based Variational Inference for Bayesian Calibration}},
author = {Exaucé Luweh Adjim Ngarti and Élise Arnaud and Léo Nicoletti and Victor Trappler and Arthur Vidard},
howpublished = {HAL},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.37169},
eprint = {2610.37169},
archivePrefix = {arXiv},
url = {https://hal.science/hal-05840257v1},
note = {CC BY-NC 4.0},
}
RIS
TY - UNPB
TI - Rényi-Regularized Flow-Based Variational Inference for Bayesian Calibration
AU - Exaucé Luweh Adjim Ngarti
AU - Élise Arnaud
AU - Léo Nicoletti
AU - Victor Trappler
AU - Arthur Vidard
PY - 2026
DA - 2026/10/27
PB - HAL
DO - 10.48550/arxiv.2610.37169
UR - https://hal.science/hal-05840257v1
AB - Variational inference with normalizing flows provides a flexible posterior approximation for Bayesian model calibration, but struggles when the posterior has several modes separated by regions of low probability mass. We study this behavior in a two-parameter twin experiment and regularize the variational objective. Under Gaussian observation errors and a Gaussian prior, the corresponding minimization objective is the expected Tikhonov calibration objective minus the differential entropy of the approximating distribution. However, the expected Tikhonov term favors concentration near one of the modes, while entropy maximization does not guarantee multimodality. We introduce a regularization term based on Rényi divergence that encourages the approximate posterior to cover the prior support. In the connected bimodal case, a flow approximation represents both modes; in the case with well-separated modes, the flow and Gaussian mixture frequently miss one mode. The introduced regularization reduces the near-collapse fraction from 50% to 25%. While the regularization improves mode coverage in this experiment, it changes the Bayesian variational objective, introduces a trade-off in approximation accuracy, and does not provide a general solution to mode collapse.
ER -