Pre-training of Bayesian Optimization Algorithm through Bayesian Optimization
Abstract
Bayesian optimization (BO) is widely used as a standard approach for expensive black-box optimization. However, BO algorithms often involve parameters that must be specified in advance, and their performance can strongly depend on these choices. We propose a framework for optimizing such parameters using sample paths drawn from a Gaussian process (GP) inferred from the information available at the start of BO. We use cumulative regret as the performance metric for a BO algorithm. By running the BO algorithm on the generated sample paths, we obtain an empirical estimate of its expected cumulative regret for a given parameter configuration. Optimizing this estimate allows us to identify parameter configurations that, given the currently available information, are expected to achieve low cumulative regret. Since this parameter optimization is itself a black-box optimization problem, we employ another BO procedure to solve it, which we refer to as outer BO. Through experiments, we demonstrate that the proposed framework can effectively select parameter configurations that achieve strong performance among a range of candidate configurations.
Related
- A sequential regularized piecewise affine algorithm for nonconvex nonsmooth multicomposite optimization in RNN training
- On the disintegration of the stochastic majority vote: from pac-bayesian bounds to a self-bounding algorithm
- Bagel: A Fast Bayesian Online Changepoint Detection Algorithm for Linear Models via Posterior Merging
- A new algorithm for topology optimization using a level-set method
- Distributed Proximal Stein Variational Gradient Descent Algorithm for Large-scale Bayesian Inference in Traveltime Tomography
BibTeX
@misc{katayama2026pretraining,
title = {{Pre-training of Bayesian Optimization Algorithm through Bayesian Optimization}},
author = {Satoshi Katayama and Shoyo Hunt and Shintaro Masuda and Masayuki Karasuyama},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.10186},
eprint = {2610.10186},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.10186},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Pre-training of Bayesian Optimization Algorithm through Bayesian Optimization
AU - Satoshi Katayama
AU - Shoyo Hunt
AU - Shintaro Masuda
AU - Masayuki Karasuyama
PY - 2026
DA - 2026/10/07
PB - arXiv
DO - 10.48550/arxiv.2610.10186
UR - https://arxiv.org/abs/2610.10186
AB - Bayesian optimization (BO) is widely used as a standard approach for expensive black-box optimization. However, BO algorithms often involve parameters that must be specified in advance, and their performance can strongly depend on these choices. We propose a framework for optimizing such parameters using sample paths drawn from a Gaussian process (GP) inferred from the information available at the start of BO. We use cumulative regret as the performance metric for a BO algorithm. By running the BO algorithm on the generated sample paths, we obtain an empirical estimate of its expected cumulative regret for a given parameter configuration. Optimizing this estimate allows us to identify parameter configurations that, given the currently available information, are expected to achieve low cumulative regret. Since this parameter optimization is itself a black-box optimization problem, we employ another BO procedure to solve it, which we refer to as outer BO. Through experiments, we demonstrate that the proposed framework can effectively select parameter configurations that achieve strong performance among a range of candidate configurations.
ER -