Predictive-Context LOO: Bayesian Model Comparison with Pretrained Predictive Learners
Abstract
Leave-one-out (LOO) cross-validation, a standard tool for Bayesian model assessment, evaluates each observation using a LOO predictive distribution derived from the model itself. In this work, we introduce an alternative construction in which posterior predictive simulation connects the Bayesian model to an external pretrained predictive learner. Our method, Predictive-Context LOO (PC-LOO), fits a Bayesian model once, generates posterior-predictive synthetic data, and uses them as context for the learner to approximate each LOO predictive density. Under an oracle construction, averaging contexts generated from the LOO posterior recovers the exact LOO predictive density. Practical PC-LOO replaces the oracle predictor by the pretrained learner and the LOO posterior by the full posterior. We quantify these errors through learner compatibility Ci and deletion influence Vi, obtaining H2 (pLOOi , qPCi) ≤ Ci + Vi. The identity also holds for any fixed subset of M context points, allowing M ≪ n − 1 as the learner's context budget, and scalability to large datasets. Experiments with TabICL support this decomposition, deletion influence decays approximately as n−1, while learner compatibility can remain limiting at large n. PC-LOO remains accurate with reduced context sizes, while model comparison accuracy is governed by differences in approximation error across models rather than their absolute errors. More broadly, PC-LOO establishes posterior predictive simulation as a principled interface for integrating pretrained predictive learners into Bayesian workflows.
Related
- Outcome-dependent density combination by mixture Bayesian predictive synthesis
- Prior Predictive Matching for Bayesian LoRA
- SCOPE Before You Sample: Conformal Scores Guide Posterior Exploration for Fast and Calibrated Bayesian Predictive Inference
- Bayesian Propensity Score Predictive Inference for Extending Trial Findings to Target Populations
- The Impact of Likelihood Tempering on the Limiting Predictive Moments of Variational Bayesian Linear Neural Networks
BibTeX
@misc{montcho2026predictivecontext,
title = {{Predictive-Context LOO: Bayesian Model Comparison with Pretrained Predictive Learners}},
author = {Hans Montcho and Maurizio Filippone},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.12688},
eprint = {2610.12688},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.12688},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Predictive-Context LOO: Bayesian Model Comparison with Pretrained Predictive Learners
AU - Hans Montcho
AU - Maurizio Filippone
PY - 2026
DA - 2026/10/08
PB - arXiv
DO - 10.48550/arxiv.2610.12688
UR - https://arxiv.org/abs/2610.12688
AB - Leave-one-out (LOO) cross-validation, a standard tool for Bayesian model assessment, evaluates each observation using a LOO predictive distribution derived from the model itself. In this work, we introduce an alternative construction in which posterior predictive simulation connects the Bayesian model to an external pretrained predictive learner. Our method, Predictive-Context LOO (PC-LOO), fits a Bayesian model once, generates posterior-predictive synthetic data, and uses them as context for the learner to approximate each LOO predictive density. Under an oracle construction, averaging contexts generated from the LOO posterior recovers the exact LOO predictive density. Practical PC-LOO replaces the oracle predictor by the pretrained learner and the LOO posterior by the full posterior. We quantify these errors through learner compatibility C_i and deletion influence V_i, obtaining H^2 (p^LOO_i , q^PC_i) ≤ C_i + V_i. The identity also holds for any fixed subset of M context points, allowing M ≪ n − 1 as the learner's context budget, and scalability to large datasets. Experiments with TabICL support this decomposition, deletion influence decays approximately as n^−1, while learner compatibility can remain limiting at large n. PC-LOO remains accurate with reduced context sizes, while model comparison accuracy is governed by differences in approximation error across models rather than their absolute errors. More broadly, PC-LOO establishes posterior predictive simulation as a principled interface for integrating pretrained predictive learners into Bayesian workflows.
ER -