Set Membership Estimation for Stochastic Linear Contextual Bandits
Abstract
Contextual bandit learning has attracted increasing attention in engineering applications due to its ability to incorporate side/context information in time-varying environments. A widely used approach for contextual bandits is optimism in the face of uncertainty (OFU), in which uncertainty sets for unknown parameters are constructed and decisions are made optimistically over these sets. In the bandit literature, uncertainty sets are typically derived using statistical learning tools, such as confidence intervals. In contrast, set membership estimation (SME), a well-established uncertainty quantification approach in control and system identification, has received little attention in the bandit literature. This paper investigates the connection between SME and contextual bandits by focusing on linear contextual bandits as a starting point. We propose an SME-based OFU algorithm and establish a sublinear regret bound under proper assumptions. Simulations further demonstrate the promising performance of SME-OFU compared with existing methods. This supplementary manuscript provides more proof details in Section IV and the Appendix.
Related
- Meta-LinEXP3: Online-within-Online Learning for Adversarial Linear Contextual Bandits
- Explore Once, Evaluate Many: Multiple Policy Evaluation for Stochastic Linear Bandits
- Diversified Multinomial Logit Contextual Bandits
- Policy Regret for Embedding Model Routing: Contextual Bandits with Low-Rank Experts
- Fast Rates for Offline Contextual Bandits with Forward-KL Regularization under Single-Policy Concentrability
BibTeX
@misc{xu2026membership,
title = {{Set Membership Estimation for Stochastic Linear Contextual Bandits}},
author = {Haonan Xu and Yingying Li},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2606.20022},
eprint = {2606.20022},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.20022},
}
RIS
TY - UNPB
TI - Set Membership Estimation for Stochastic Linear Contextual Bandits
AU - Haonan Xu
AU - Yingying Li
PY - 2026
DA - 2026/10/15
PB - arXiv
DO - 10.48550/arxiv.2606.20022
UR - https://arxiv.org/abs/2606.20022
AB - Contextual bandit learning has attracted increasing attention in engineering applications due to its ability to incorporate side/context information in time-varying environments. A widely used approach for contextual bandits is optimism in the face of uncertainty (OFU), in which uncertainty sets for unknown parameters are constructed and decisions are made optimistically over these sets. In the bandit literature, uncertainty sets are typically derived using statistical learning tools, such as confidence intervals. In contrast, set membership estimation (SME), a well-established uncertainty quantification approach in control and system identification, has received little attention in the bandit literature. This paper investigates the connection between SME and contextual bandits by focusing on linear contextual bandits as a starting point. We propose an SME-based OFU algorithm and establish a sublinear regret bound under proper assumptions. Simulations further demonstrate the promising performance of SME-OFU compared with existing methods. This supplementary manuscript provides more proof details in Section IV and the Appendix.
ER -