Exploration and Experimentation with a Misspecified Simulator
Abstract
Randomized experiments are the standard for measuring causal effects that guide decisions, but are costly. Simulators calibrated to historical data offer a cheaper alternative, but their policies can become suboptimal under drift, a change in dynamics after calibration. We study if, when, and how to collect data for recalibrating the simulator. We consider a system with multiple units running in parallel and managed by a single operator. The real-world system is a partially observed process with a latent variable, while the simulator models it as a fully observed Markovian process, and thus is misspecified. Two data collection strategies exploit the current policy: (i) never recalibrating and (ii) recalibrating from on-policy data (passive adaptation). Two others collect off-policy data by trying actions outside the current policy: (iii) continuous exploration across all units and (iv) randomized experimentation on a sample of units over a predefined window. We investigate these strategies theoretically and in a vending-machine simulation. Experiments can improve on passive adaptation when drift is invisible in on-policy data or recalibration lags behind continuing drift. Experimentation is preferable to exploration when trying a worse action is expensive and the dynamics may not have changed.
Related
- Finite-Sample LTI System Identification from a Misspecified Finite Model Set
- Sign Betting for Misspecified Regression: Variance Adaptivity and Optimal Approximation
- Repairing Locally Misspecified GMM: An Empirical Bayes Approach
- Modular Bayesian Inference for Partially Misspecified Mixtures with Uncertain Class Labels
- On the Optimality of Random Features under General Conditions: Misspecified Cases and Robust Learning
BibTeX
@misc{parikh2026exploration,
title = {{Exploration and Experimentation with a Misspecified Simulator}},
author = {Harsh Parikh and Gabriel Levin-Konigsberg and Sohrab Andaz and Akhil Bagaria and Dominique Perrault-Joncas and Alexander Volfovsky},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.21458},
eprint = {2605.21458},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.21458},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Exploration and Experimentation with a Misspecified Simulator
AU - Harsh Parikh
AU - Gabriel Levin-Konigsberg
AU - Sohrab Andaz
AU - Akhil Bagaria
AU - Dominique Perrault-Joncas
AU - Alexander Volfovsky
PY - 2026
DA - 2026/10/16
PB - arXiv
DO - 10.48550/arxiv.2605.21458
UR - https://arxiv.org/abs/2605.21458
AB - Randomized experiments are the standard for measuring causal effects that guide decisions, but are costly. Simulators calibrated to historical data offer a cheaper alternative, but their policies can become suboptimal under drift, a change in dynamics after calibration. We study if, when, and how to collect data for recalibrating the simulator. We consider a system with multiple units running in parallel and managed by a single operator. The real-world system is a partially observed process with a latent variable, while the simulator models it as a fully observed Markovian process, and thus is misspecified. Two data collection strategies exploit the current policy: (i) never recalibrating and (ii) recalibrating from on-policy data (passive adaptation). Two others collect off-policy data by trying actions outside the current policy: (iii) continuous exploration across all units and (iv) randomized experimentation on a sample of units over a predefined window. We investigate these strategies theoretically and in a vending-machine simulation. Experiments can improve on passive adaptation when drift is invisible in on-policy data or recalibration lags behind continuing drift. Experimentation is preferable to exploration when trying a worse action is expensive and the dynamics may not have changed.
ER -