Prior-Data Fitted Networks for In-Context Off-Policy Evaluation
Abstract
Off-policy evaluation (OPE) is essential for assessing decision policies before deployment when online experimentation is costly or unsafe. Accurate estimation remains difficult when behavior-policy data are limited and the target policy favors states and actions rarely observed in those data. We propose InCOPE, a prior-data fitted network that learns a shared policy-value prediction rule across synthetic OPE tasks. We construct an OPE task prior over Markov decision processes and behavior-target policy pairs, generating behavior-policy datasets and exact target-policy values for pretraining. For a new tabular decision problem, the pretrained network uses the behavior-policy dataset, target policy, and known task settings as context to predict the policy's expected return without task-specific parameter updates. Experiments across four tabular domains, including a semi-synthetic clinical setting, demonstrate InCOPE's effectiveness for policy-value estimation.
Related
- Prior-data fitted networks for variable selection
- How Well Do Prior-Data Fitted Networks Amortize Bayesian Inference?
- Anisotropic Trust Region Warping for Prior-Data Fitted Networks in Bayesian Optimisation
- Characterizing the Separability Gap in Prior-Data Fitted Networks
- Amortized Nonlinear Variable Selection with Prior-Data Fitted Networks
BibTeX
@misc{yu2026priordata,
title = {{Prior-Data Fitted Networks for In-Context Off-Policy Evaluation}},
author = {Shuguang Yu and Ting Li and Fan Zhou},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.34035},
eprint = {2610.34035},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.34035},
}
RIS
TY - UNPB
TI - Prior-Data Fitted Networks for In-Context Off-Policy Evaluation
AU - Shuguang Yu
AU - Ting Li
AU - Fan Zhou
PY - 2026
DA - 2026/10/25
PB - arXiv
DO - 10.48550/arxiv.2610.34035
UR - https://arxiv.org/abs/2610.34035
AB - Off-policy evaluation (OPE) is essential for assessing decision policies before deployment when online experimentation is costly or unsafe. Accurate estimation remains difficult when behavior-policy data are limited and the target policy favors states and actions rarely observed in those data. We propose InCOPE, a prior-data fitted network that learns a shared policy-value prediction rule across synthetic OPE tasks. We construct an OPE task prior over Markov decision processes and behavior-target policy pairs, generating behavior-policy datasets and exact target-policy values for pretraining. For a new tabular decision problem, the pretrained network uses the behavior-policy dataset, target policy, and known task settings as context to predict the policy's expected return without task-specific parameter updates. Experiments across four tabular domains, including a semi-synthetic clinical setting, demonstrate InCOPE's effectiveness for policy-value estimation.
ER -