Optimal approximation error approach in Bayesian inverse problems
Abstract
In many inverse problems, evaluating the forward operator is computationally expensive, making it common practice to accelerate inference with a less costly approximate model. However, such approximations introduce unwanted bias and must be carefully accounted for to ensure the reliability of the solution. This phenomenon is particularly pronounced in Bayesian inference, where accurate posterior characterisation typically requires a large number of forward model evaluations. In this paper, we focus on the approximation error model (AEM), which compensates for the modelling error by adjusting the likelihood density. More precisely, AEM incorporates the model error into the additive Gaussian noise distribution. A central challenge in this approach is how to choose the modified noise mean and covariance in a principled way. To address this, we propose a new framework, optimal approximation error (OAE) which determines the noise model by minimising the expected information loss, measured via the Kullback-Leibler (KL) divergence between the true and approximate posterior distributions. We show that the standard enhanced error approach (EEA) is recovered as the minimiser of the expected divergence between likelihoods rather than posteriors and develop OAE as a direct alternative targeting posterior accuracy. We establish bi-convexity of the OAE objective, introduce a regularised, provably convergent proximal alternating minimisation scheme, and derive closed-form expressions in the Gaussian linear setting. We further show that OAE estimates computed from empirical training data are statistically consistent. Numerical experiments on a linear CT problem and a nonlinear Darcy-flow problem show that OAE improves posterior accuracy relative to both the uncorrected and EEA-corrected likelihoods.
Related
- Sequential Entropic Optimal Transport for Bayesian Inverse Problems via Multilevel Monte Carlo
- WEAKLY VISCOUS NONLOCAL WAVE EQUATIONS: FORWARD THEORY, LEBESGUE RUNGE APPROXIMATION, AND INVERSE PROBLEMS
- Accelerating posterior sampling in Bayesian inverse problems with diffusion models: an online-offline perspective
- WEAKLY VISCOUS NONLOCAL WAVE EQUATIONS WITH VARIABLE COEFFICIENTS: WELL-POSEDNESS, RUNGE APPROXIMATION, AND INVERSE PROBLEMS
- PDE-based Bayesian Inverse Problems in All-at-once Framework
BibTeX
@misc{schneider2026optimal,
title = {{Optimal approximation error approach in Bayesian inverse problems}},
author = {Fabian Schneider and Tapio Helin},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.38434},
eprint = {2610.38434},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.38434},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Optimal approximation error approach in Bayesian inverse problems
AU - Fabian Schneider
AU - Tapio Helin
PY - 2026
DA - 2026/10/28
PB - arXiv
DO - 10.48550/arxiv.2610.38434
UR - https://arxiv.org/abs/2610.38434
AB - In many inverse problems, evaluating the forward operator is computationally expensive, making it common practice to accelerate inference with a less costly approximate model. However, such approximations introduce unwanted bias and must be carefully accounted for to ensure the reliability of the solution. This phenomenon is particularly pronounced in Bayesian inference, where accurate posterior characterisation typically requires a large number of forward model evaluations. In this paper, we focus on the approximation error model (AEM), which compensates for the modelling error by adjusting the likelihood density. More precisely, AEM incorporates the model error into the additive Gaussian noise distribution. A central challenge in this approach is how to choose the modified noise mean and covariance in a principled way. To address this, we propose a new framework, optimal approximation error (OAE) which determines the noise model by minimising the expected information loss, measured via the Kullback-Leibler (KL) divergence between the true and approximate posterior distributions. We show that the standard enhanced error approach (EEA) is recovered as the minimiser of the expected divergence between likelihoods rather than posteriors and develop OAE as a direct alternative targeting posterior accuracy. We establish bi-convexity of the OAE objective, introduce a regularised, provably convergent proximal alternating minimisation scheme, and derive closed-form expressions in the Gaussian linear setting. We further show that OAE estimates computed from empirical training data are statistically consistent. Numerical experiments on a linear CT problem and a nonlinear Darcy-flow problem show that OAE improves posterior accuracy relative to both the uncorrected and EEA-corrected likelihoods.
ER -