Model Criticism via Stochastic Optimization
Abstract
Many generative models are specified by how data are simulated. An observation is a deterministic transform Y=G(θ,X) of a draw X from an assumed noise law, indexed by a structural parameter θ. Such models cover discrete, continuous and functional data, as well as i.i.d. replicates of stochastic processes, but their likelihoods are often unavailable. We propose a simulation-based procedure that tests whether such a model fits the data and, when it does not, locates where it fails, without choosing a test statistic. The procedure splits the data in two and is formulated as a two-stage stochastic program. In the estimation stage, θ is fitted on the first block by minimizing a Wasserstein distance between the data and simulations from the model. In the recourse stage, θ is frozen at the fitted θ̃, and a finite set of representative noise values {x̅j} is pushed through the fitted mechanism to give model atoms G(θ̃,x̅j). The held-out block optimizes weights w on the model atoms to best match its own distribution. The optimized weights are then compared with theoretical weights under correct specification. Since each model atom is the image of known noise values, the disagreement between optimal and theoretical weights points to the part of the noise law that is wrong. The split makes the scaled weights conditionally multinomial, yielding a calibrated Pearson test. We prove strong consistency of both stages under checkable assumptions and illustrate the procedure on a Gaussian location--scale family, the number of segregating sites under Kingman's coalescent, a Yule process, and an Ornstein--Uhlenbeck diffusion.
Related
- Stochastic Optimization and Data Science
- Stochastic Optimization of Tree Tensor Networks
- Random tilts to find stationary points in stochastic convex optimization
- Gradient-Free Methods for Stochastic Convex Optimization with Stochastic Functional Constraints
- Accelerated Stochastic Zeroth-Order Quasar-Convex Optimization
BibTeX
@misc{wang2026model,
title = {{Model Criticism via Stochastic Optimization}},
author = {Ren-Yi Wang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.37837},
eprint = {2610.37837},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.37837},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Model Criticism via Stochastic Optimization
AU - Ren-Yi Wang
PY - 2026
DA - 2026/10/27
PB - arXiv
DO - 10.48550/arxiv.2610.37837
UR - https://arxiv.org/abs/2610.37837
AB - Many generative models are specified by how data are simulated. An observation is a deterministic transform Y=G(θ,X) of a draw X from an assumed noise law, indexed by a structural parameter θ. Such models cover discrete, continuous and functional data, as well as i.i.d. replicates of stochastic processes, but their likelihoods are often unavailable. We propose a simulation-based procedure that tests whether such a model fits the data and, when it does not, locates where it fails, without choosing a test statistic. The procedure splits the data in two and is formulated as a two-stage stochastic program. In the estimation stage, θ is fitted on the first block by minimizing a Wasserstein distance between the data and simulations from the model. In the recourse stage, θ is frozen at the fitted θ̃, and a finite set of representative noise values {x̅_j} is pushed through the fitted mechanism to give model atoms G(θ̃,x̅_j). The held-out block optimizes weights w on the model atoms to best match its own distribution. The optimized weights are then compared with theoretical weights under correct specification. Since each model atom is the image of known noise values, the disagreement between optimal and theoretical weights points to the part of the noise law that is wrong. The split makes the scaled weights conditionally multinomial, yielding a calibrated Pearson test. We prove strong consistency of both stages under checkable assumptions and illustrate the procedure on a Gaussian location--scale family, the number of segregating sites under Kingman's coalescent, a Yule process, and an Ornstein--Uhlenbeck diffusion.
ER -