From Frequentist to Bayesian Contextual Optimization
Abstract
In data-driven contextual stochastic optimization, existing approaches are predominantly frequentist: they commit to a single predictive model and treat it as ground truth, yielding prescriptions that are fragile to model uncertainty and sampling variability in small- and moderate-sample regimes. We propose Bayesian contextual optimization (BCO), a framework that maintains a Gibbs posterior over the parameter space. This decision-focused posterior weights candidate models by their empirical decision quality rather than statistical fit, thereby avoiding commitment to a potentially misspecified likelihood while encoding the full distribution of plausible models consistent with the data. A Bayesian contextual policy is then derived by minimizing the expected decision cost under the posterior predictive distribution, hedging prescriptions against model uncertainty by aggregating over the posterior. We establish three theoretical guarantees: (i) the Gibbs posterior concentrates exponentially fast around the frequentist best-in-class parameter set; (ii) BCO can strictly improve over the frequentist best-in-class policy when model uncertainty is non-negligible; and (iii) BCO attains an O(n−1/2) excess risk rate against the oracle over all probability measures up to a misspecification term and an oracle aggregate gap. Computationally, we tailor a variational inference scheme that has the same per-iteration cost as frequentist alternatives and a gradient-free Metropolis-Hastings algorithm that handles nondifferentiable problems. Numerical experiments on two-stage shipment planning, contextual newsvendor, and return-constrained portfolio problems confirm that BCO consistently reduces out-of-sample cost and variance relative to kernel estimators and decision-focused baselines, with the most pronounced gains in small- and moderate-sample regimes under substantial model uncertainty.
Related
- Inconsistency between Frequentist and Bayesian inference, approximation of improper priors and permissible priors
- Frequentist and Bayesian population pharmacometrics: similarities, differences, and practical implications
- The Curvature of Regret in Contextual Linear Optimization
- Preventing Model Extraction in Contextual Linear Optimization
- Largest Rashomon sets of decision trees for robust contextual optimization
BibTeX
@misc{xie2026from,
title = {{From Frequentist to Bayesian Contextual Optimization}},
author = {Zhuojun Xie and Adam Abdin and Yiping Fang},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.27534},
eprint = {2609.27534},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.27534},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - From Frequentist to Bayesian Contextual Optimization
AU - Zhuojun Xie
AU - Adam Abdin
AU - Yiping Fang
PY - 2026
DA - 2026/09/23
PB - arXiv
DO - 10.48550/arxiv.2609.27534
UR - https://arxiv.org/abs/2609.27534
AB - In data-driven contextual stochastic optimization, existing approaches are predominantly frequentist: they commit to a single predictive model and treat it as ground truth, yielding prescriptions that are fragile to model uncertainty and sampling variability in small- and moderate-sample regimes. We propose Bayesian contextual optimization (BCO), a framework that maintains a Gibbs posterior over the parameter space. This decision-focused posterior weights candidate models by their empirical decision quality rather than statistical fit, thereby avoiding commitment to a potentially misspecified likelihood while encoding the full distribution of plausible models consistent with the data. A Bayesian contextual policy is then derived by minimizing the expected decision cost under the posterior predictive distribution, hedging prescriptions against model uncertainty by aggregating over the posterior. We establish three theoretical guarantees: (i) the Gibbs posterior concentrates exponentially fast around the frequentist best-in-class parameter set; (ii) BCO can strictly improve over the frequentist best-in-class policy when model uncertainty is non-negligible; and (iii) BCO attains an O(n^−1/2) excess risk rate against the oracle over all probability measures up to a misspecification term and an oracle aggregate gap. Computationally, we tailor a variational inference scheme that has the same per-iteration cost as frequentist alternatives and a gradient-free Metropolis-Hastings algorithm that handles nondifferentiable problems. Numerical experiments on two-stage shipment planning, contextual newsvendor, and return-constrained portfolio problems confirm that BCO consistently reduces out-of-sample cost and variance relative to kernel estimators and decision-focused baselines, with the most pronounced gains in small- and moderate-sample regimes under substantial model uncertainty.
ER -