Learning rate selection via weighted Fisher divergence
Abstract
The general Bayesian approach provides a flexible framework by introducing a loss-based likelihood with a learning rate controlling the relative weight of the likelihood and prior. This flexibility creates an important calibration problem under model misspecification, where the conventional Bayesian information identity fails and uncalibrated Gibbs posteriors may not provide reliable uncertainty quantification. We propose a method for selecting the learning rate by minimizing a weighted Fisher divergence between the asymptotic general posterior and a normal distribution with sandwich-type covariance, yielding a closed-form calibrated learning rate. The proposed framework includes Fisher information matching as a special case and, for an important choice of the weight matrix, yields a learning rate no larger than the Fisher information matching rate. This leads to a more dispersed posterior and can reduce the risk of undercoverage. The framework also includes a learning rate based on Kullback--Leibler divergence as a special case. Numerical examples show that the proposed method tends to mitigate undercoverage under model misspecification, while real-data analyses illustrate its practical applicability.
Related
- Logarithmic upper bounds and axial front selection for spatially periodic lattice Fisher--KPP equations
- Weighted selection from elliptical distributions: a stochastic representation and an application to portfolio separation
- Weighted Data Selection: Sharp Upper-Half and Five-Dimensional Laws
- An Inverse-Selection-Probability Weighted Approach for Real-World Progression-Free Survival With Left-Truncated Clinico-Genomic Data.
- Finite inverse nodal problems for singular weighted Sturm--Liouville equations: variational selection and spectral matching
BibTeX
@misc{onizuka2026learning,
title = {{Learning rate selection via weighted Fisher divergence}},
author = {Takahiro Onizuka},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2606.26478},
eprint = {2606.26478},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.26478},
}
RIS
TY - UNPB
TI - Learning rate selection via weighted Fisher divergence
AU - Takahiro Onizuka
PY - 2026
DA - 2026/09/20
PB - arXiv
DO - 10.48550/arxiv.2606.26478
UR - https://arxiv.org/abs/2606.26478
AB - The general Bayesian approach provides a flexible framework by introducing a loss-based likelihood with a learning rate controlling the relative weight of the likelihood and prior. This flexibility creates an important calibration problem under model misspecification, where the conventional Bayesian information identity fails and uncalibrated Gibbs posteriors may not provide reliable uncertainty quantification. We propose a method for selecting the learning rate by minimizing a weighted Fisher divergence between the asymptotic general posterior and a normal distribution with sandwich-type covariance, yielding a closed-form calibrated learning rate. The proposed framework includes Fisher information matching as a special case and, for an important choice of the weight matrix, yields a learning rate no larger than the Fisher information matching rate. This leads to a more dispersed posterior and can reduce the risk of undercoverage. The framework also includes a learning rate based on Kullback--Leibler divergence as a special case. Numerical examples show that the proposed method tends to mitigate undercoverage under model misspecification, while real-data analyses illustrate its practical applicability.
ER -