Sharp Characterization of Bias in Post-Bandit Inference
Abstract
Bandit algorithms generate data for downstream inference, but adaptive sampling biases post-bandit sample means. We analyze this bias for stable index algorithms, including UCB1 and its generalizations, and derive sharp leading-order expressions for the sample-mean bias and expected Z-statistic, in bandit experiments of fixed horizon T. Our characterization reveals the algorithmic origin of bias through a key index-function-dependent quantity, which we term effective exploration rate. For example, under UCB1, the effective exploration rate is of order √(log T), and the standardized bias of any arm (that is not uniquely optimal) decays at the extremely slow rate 1/√(log T). We also show how the choice of the index function affects both regret and bias, which reveals a regret-bias trade-off: more exploratory algorithm reduces bias but increases regret. We further show how bias most severely distorts confidence intervals and hypothesis tests when the tested arm is one of the tied-optimal arms. Our sharp characterization for bias uses a novel empirical fluid approximation of the algorithm's sampling dynamics, which may be of independent interest.
Related
- Exploration strategies for Gaussian process bandit optimization with inference regret guarantees
- Bandit Process Inference for Streaming Functional Data
- PAC Learning with Bandit Feedback: Sharp Sample Complexity in the Realizable Setting
- Sharp Bounds on the Winner's Bias under Independent and Nonnegatively Correlated Noise
- Bias-robust causal inference for panel data
BibTeX
@misc{wang2026sharp,
title = {{Sharp Characterization of Bias in Post-Bandit Inference}},
author = {Lisu Wang and Yilun Chen and Jiaqi Lu},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2608.01069},
eprint = {2608.01069},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.01069},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Sharp Characterization of Bias in Post-Bandit Inference
AU - Lisu Wang
AU - Yilun Chen
AU - Jiaqi Lu
PY - 2026
DA - 2026/09/22
PB - arXiv
DO - 10.48550/arxiv.2608.01069
UR - https://arxiv.org/abs/2608.01069
AB - Bandit algorithms generate data for downstream inference, but adaptive sampling biases post-bandit sample means. We analyze this bias for stable index algorithms, including UCB1 and its generalizations, and derive sharp leading-order expressions for the sample-mean bias and expected Z-statistic, in bandit experiments of fixed horizon T. Our characterization reveals the algorithmic origin of bias through a key index-function-dependent quantity, which we term effective exploration rate. For example, under UCB1, the effective exploration rate is of order √(log T), and the standardized bias of any arm (that is not uniquely optimal) decays at the extremely slow rate 1/√(log T). We also show how the choice of the index function affects both regret and bias, which reveals a regret-bias trade-off: more exploratory algorithm reduces bias but increases regret. We further show how bias most severely distorts confidence intervals and hypothesis tests when the tested arm is one of the tied-optimal arms. Our sharp characterization for bias uses a novel empirical fluid approximation of the algorithm's sampling dynamics, which may be of independent interest.
ER -