Simulation-based Criticism of Complex Biological Models with Neural Evidence Estimation
Abstract
In statistical modeling, the likelihood is the basis for both inference and criticism, which constitute the two stages of data analysis. However, many realistic scientific models lack computationally tractable likelihoods. Increasingly, these are implemented as computer simulations. New deep learning-based methods for simulation-based inference (SBI) have demonstrated the ability to efficiently and accurately calibrate complex simulation models. However, in this setting, the standard tools for comparing models and assessing their potential limitations are often unavailable. We propose a new method for simulation-based model criticism, Neural Evidence Estimation. NEE trains a normalizing flow on simulated data to learn an estimate of the marginal likelihood (a.k.a. evidence). These estimates can be used to diagnose model misspecification and perform model comparison without evaluating (or approximating) a model's likelihood. Compared to related techniques designed for either of these tasks, NEE is quick, robust, and modular. We demonstrate our methodology through applications to infectious disease modeling. Here, model criticism is difficult yet imperative: infectious disease modelers often face a choice among several plausible models featuring intractable likelihoods. We first validate NEE's ability to discern incorrect transmission models through simulation experiments. We then perform data analysis of a historic multi-wave influenza epidemic, where we use SBI and NEE to rigorously compare several competing mechanistic theories of reinfection. Overall, NEE provides a unifying framework for simulation-based criticism, advances efficient multi-model data analysis, and helps improve the reliability of SBI.
Related
- Numerical simulation of biological capsules based on interface-fitted polygonal meshes
- Simulation-consistent Estimation of the Marginal Likelihood for Block Models
- Estimation, Sensitivity Analysis, and Optimization of Distortion Risk Measures via Simulation
- Neural composite likelihood estimation: simulation based inference for time series
- Clinically Interpretable Biological Age Estimation Using CatBoost and SHAP with Continuous and Categorical Features from NHANES
BibTeX
@misc{chatha2026simulationbased,
title = {{Simulation-based Criticism of Complex Biological Models with Neural Evidence Estimation}},
author = {Prayag Chatha and Fan Bu and Jon Zelner},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2609.22563},
eprint = {2609.22563},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.22563},
}
RIS
TY - UNPB
TI - Simulation-based Criticism of Complex Biological Models with Neural Evidence Estimation
AU - Prayag Chatha
AU - Fan Bu
AU - Jon Zelner
PY - 2026
DA - 2026/10/26
PB - arXiv
DO - 10.48550/arxiv.2609.22563
UR - https://arxiv.org/abs/2609.22563
AB - In statistical modeling, the likelihood is the basis for both inference and criticism, which constitute the two stages of data analysis. However, many realistic scientific models lack computationally tractable likelihoods. Increasingly, these are implemented as computer simulations. New deep learning-based methods for simulation-based inference (SBI) have demonstrated the ability to efficiently and accurately calibrate complex simulation models. However, in this setting, the standard tools for comparing models and assessing their potential limitations are often unavailable. We propose a new method for simulation-based model criticism, Neural Evidence Estimation. NEE trains a normalizing flow on simulated data to learn an estimate of the marginal likelihood (a.k.a. evidence). These estimates can be used to diagnose model misspecification and perform model comparison without evaluating (or approximating) a model's likelihood. Compared to related techniques designed for either of these tasks, NEE is quick, robust, and modular. We demonstrate our methodology through applications to infectious disease modeling. Here, model criticism is difficult yet imperative: infectious disease modelers often face a choice among several plausible models featuring intractable likelihoods. We first validate NEE's ability to discern incorrect transmission models through simulation experiments. We then perform data analysis of a historic multi-wave influenza epidemic, where we use SBI and NEE to rigorously compare several competing mechanistic theories of reinfection. Overall, NEE provides a unifying framework for simulation-based criticism, advances efficient multi-model data analysis, and helps improve the reliability of SBI.
ER -