What a Data Release Must Preserve for Off-Policy Evaluation
Abstract
In off-policy evaluation, merging decision-time histories that matter differently for a specified policy value or comparison can make that evaluation unidentified, even with observed actions and outcomes and known behavior and target policies. We characterize which distinctions must be preserved: identification holds if and only if the action-specific evaluation weights are constant within each released record, and the coarsest exact release groups histories by these weights. For normalized outcomes, the largest sharp width induced by the release equals a behavior-weighted within-record radius of these weights. The same radius becomes a release-design objective when each record may be refined into at most m categories. The optimum has an exact m-center characterization. When behavior does not vary within a released record, we give a finite-data repair algorithm whose maximum evaluation-weight radius is at most twice the finite-data optimum. For a fixed action-outcome distribution within a released record, a distinct sharp-identification problem remains. Under the same condition, the action-specific outcome distributions are identified, and optimizing over their couplings gives the sharp interval. We construct an AIPW envelope containing this sharp interval; it matches the interval for all such outcome distributions exactly when the extreme within-record action-coefficient vectors take at most two values in each action coordinate. In 24 KuaiRec tasks, the repair algorithm yields a 93% median reduction in mean pairwise sharp width at m=64 relative to the unrepaired release.
Related
- When Can We Work in Embedding Space? What Text Embeddings Preserve
- The Invocation Bottleneck in LLM-Based Lossless Compression: What Can Be Removed and What Must Be Traded
- What the Machine Invented: Mining New Mathematical Instruments from OpenAI's October 2026 Release of AI-Generated Mathematics
- Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models
- Reinforcement Learning Under Runtime Confounding: Policy Learning and Off-Policy Evaluation
BibTeX
@misc{lee2026what,
title = {{What a Data Release Must Preserve for Off-Policy Evaluation}},
author = {Kyungbok Lee and Yating Zou and Michael R. Kosorok},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.15283},
eprint = {2610.15283},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.15283},
}
RIS
TY - UNPB
TI - What a Data Release Must Preserve for Off-Policy Evaluation
AU - Kyungbok Lee
AU - Yating Zou
AU - Michael R. Kosorok
PY - 2026
DA - 2026/10/11
PB - arXiv
DO - 10.48550/arxiv.2610.15283
UR - https://arxiv.org/abs/2610.15283
AB - In off-policy evaluation, merging decision-time histories that matter differently for a specified policy value or comparison can make that evaluation unidentified, even with observed actions and outcomes and known behavior and target policies. We characterize which distinctions must be preserved: identification holds if and only if the action-specific evaluation weights are constant within each released record, and the coarsest exact release groups histories by these weights. For normalized outcomes, the largest sharp width induced by the release equals a behavior-weighted within-record radius of these weights. The same radius becomes a release-design objective when each record may be refined into at most m categories. The optimum has an exact m-center characterization. When behavior does not vary within a released record, we give a finite-data repair algorithm whose maximum evaluation-weight radius is at most twice the finite-data optimum. For a fixed action-outcome distribution within a released record, a distinct sharp-identification problem remains. Under the same condition, the action-specific outcome distributions are identified, and optimizing over their couplings gives the sharp interval. We construct an AIPW envelope containing this sharp interval; it matches the interval for all such outcome distributions exactly when the extreme within-record action-coefficient vectors take at most two values in each action coordinate. In 24 KuaiRec tasks, the repair algorithm yields a 93% median reduction in mean pairwise sharp width at m=64 relative to the unrepaired release.
ER -