Log predictive density score estimation in the presence of outliers
Abstract
The log predictive density score (LPDS) and related estimators of the expected log predictive density (ELPD) have become the de facto standard for Bayesian model comparison. The LPDS is a natural out-of-sample measure of predictive density performance that accounts for parameter uncertainty, is less dependent on the prior than the marginal likelihood, and is easily computed sequentially using standard Monte Carlo from posterior draws. Our article demonstrates, however, that the standard Monte Carlo estimate of the LPDS can be extremely variable and severely biased when the test data has outliers, leading to misleading conclusions and suboptimal model choices. Estimating the LPDS as the difference between the log marginal likelihood of the full dataset and the training data is shown to be a dramatically more efficient approach. This estimator is also shown to be a much more effective fallback for observations that have been flagged as problematic by the widely used PSIS-LOO model comparison framework in Stan.
Related
- Weighted Score Matching on Bounded Domains: Nonparametric Density Estimation and Interaction Graph Recovery
- Amortized Bandwidth Learning for Kernel Density Estimation under Logarithmic Score
- Copula Active Subspaces I: A Score-Covariance Method for Reduced-Order Non-Gaussian Density Estimation
- Simultaneous comparison of the predictive values of two binary diagnostic tests in the presence of categorical covariates
- Optimal Hamiltonian Parameter Estimation in the Presence of Nuisance Parameters
BibTeX
@misc{gustafsson2026predictive,
title = {{Log predictive density score estimation in the presence of outliers}},
author = {Oskar Gustafsson and Mattias Villani},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.04748},
eprint = {2610.04748},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.04748},
}
RIS
TY - UNPB
TI - Log predictive density score estimation in the presence of outliers
AU - Oskar Gustafsson
AU - Mattias Villani
PY - 2026
DA - 2026/10/03
PB - arXiv
DO - 10.48550/arxiv.2610.04748
UR - https://arxiv.org/abs/2610.04748
AB - The log predictive density score (LPDS) and related estimators of the expected log predictive density (ELPD) have become the de facto standard for Bayesian model comparison. The LPDS is a natural out-of-sample measure of predictive density performance that accounts for parameter uncertainty, is less dependent on the prior than the marginal likelihood, and is easily computed sequentially using standard Monte Carlo from posterior draws. Our article demonstrates, however, that the standard Monte Carlo estimate of the LPDS can be extremely variable and severely biased when the test data has outliers, leading to misleading conclusions and suboptimal model choices. Estimating the LPDS as the difference between the log marginal likelihood of the full dataset and the training data is shown to be a dramatically more efficient approach. This estimator is also shown to be a much more effective fallback for observations that have been flagged as problematic by the widely used PSIS-LOO model comparison framework in Stan.
ER -