The Predictive--Optimization Gap in GuanDan Self-Play
Abstract
Outcome-predictive signals are often treated as promising optimization targets in reinforcement learning, but predictive validity and optimization utility are different claims. We expose this distinction in GuanDan, a multi-round imperfect-information game in which the implication of a round outcome depends on the public match state. Held-out analysis shows that state--gain interactions provide incremental predictive value beyond a constant signed-gain effect, establishing that match progress is predictively meaningful. We then use changes in a frozen match-win predictor, ΔΦ, as a controlled round-level supervision intervention in a matched DanZero-style Monte Carlo self-play protocol. The intervention can succeed in an individual lineage: a selected progress-augmented checkpoint substantially outperforms its matched Baseline on a locked evaluation. Yet the method-level result reverses this impression: none of five preregistered replication seeds is positive, the pooled ten-seed interval includes zero, and across-iteration effects repeatedly change sign without a common late-stage advantage. These results identify a predictive--optimization gap: under the studied learner, a signal can describe outcome-relevant progress while failing to provide a reliably beneficial optimization direction.
Related
- A Contrast-Source Inversion Scheme Based on Stochastic Optimization and Plug-and-Play Regularization
- Opening Rankings from Engine Self-Play Are Less Certain Than They Look: Sampling Error, Draws, and Strength Dependence in Strength-Limited Stockfish Games
- Towards Sustainable Hydrogen Systems: Supply Chain Optimization with Model Predictive Control and Reinforcement Learning
- JEPADepth: Self-Supervised Monocular Depth Estimation via Joint-Embedding Predictive Architectures
- Distributed electricity–carbon co-optimization of hydrogen energy storage microgrids with dynamic laplace perturbation and model predictive control
BibTeX
@misc{zou2026predictiveoptimization,
title = {{The Predictive--Optimization Gap in GuanDan Self-Play}},
author = {Qicheng Zou and Biao Yuan and Siyu Song and Wentao Liu and Xiaokun Wang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.29411},
eprint = {2610.29411},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.29411},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - The Predictive--Optimization Gap in GuanDan Self-Play
AU - Qicheng Zou
AU - Biao Yuan
AU - Siyu Song
AU - Wentao Liu
AU - Xiaokun Wang
PY - 2026
DA - 2026/10/21
PB - arXiv
DO - 10.48550/arxiv.2610.29411
UR - https://arxiv.org/abs/2610.29411
AB - Outcome-predictive signals are often treated as promising optimization targets in reinforcement learning, but predictive validity and optimization utility are different claims. We expose this distinction in GuanDan, a multi-round imperfect-information game in which the implication of a round outcome depends on the public match state. Held-out analysis shows that state--gain interactions provide incremental predictive value beyond a constant signed-gain effect, establishing that match progress is predictively meaningful. We then use changes in a frozen match-win predictor, ΔΦ, as a controlled round-level supervision intervention in a matched DanZero-style Monte Carlo self-play protocol. The intervention can succeed in an individual lineage: a selected progress-augmented checkpoint substantially outperforms its matched Baseline on a locked evaluation. Yet the method-level result reverses this impression: none of five preregistered replication seeds is positive, the pooled ten-seed interval includes zero, and across-iteration effects repeatedly change sign without a common late-stage advantage. These results identify a predictive--optimization gap: under the studied learner, a signal can describe outcome-relevant progress while failing to provide a reliably beneficial optimization direction.
ER -