Anisotropic Trust Region Warping for Prior-Data Fitted Networks in Bayesian Optimisation
Abstract
Prior-Data Fitted Networks (PFNs) replace the hyperparameter refitting of Gaussian Process (GP) surrogates in Bayesian optimisation with a single forward pass, at the cost of a prior that is fixed at meta-training time. We show empirically that a PFN's predictive log-likelihood falls increasingly behind that of a refitted GP as the objective function's lengthscale grows beyond the mode of the PFN's meta-training prior, and that this deficit carries over to optimisation performance. To correct the mismatch without retraining the network, we introduce Anisotropic Trust Region (ATR) warping. Per-dimension lengthscales are estimated by kernel target alignment, and an affine rescaling of the input space maps the objective onto the lengthscale the PFN expects, with a trust region around the anchor determining which observations are retained, leaving the PFN's 𝒪(d · N2) inference cost unchanged. On functions drawn from a Matérn-3/2 kernel, ATR warping improves optimisation performance of the PFN at both extremes of the lengthscale range. Our implementation is publicly available at https://github.com/youngers2006/Trust-Region-Warping-For-PFNs.
Related
- Optimisation of distributed storage in existing district heating networks to solve hydraulic bottlenecks
- A MINLP optimisation method for the integration of storage in district heating networks
- PKComb-BOIN12: A Pharmacokinetically Guided Bayesian Design for Dose Optimisation in Oncology Drug-Combination Trials
- Evaluation of optimisation and Bayesian inference methods for reaction rates in atmospheric chemical mechanisms
- Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisation
BibTeX
@misc{younger2026anisotropic,
title = {{Anisotropic Trust Region Warping for Prior-Data Fitted Networks in Bayesian Optimisation}},
author = {Samuel Younger and Tinkle Chugh and George De Ath},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.37737},
eprint = {2610.37737},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.37737},
}
RIS
TY - UNPB
TI - Anisotropic Trust Region Warping for Prior-Data Fitted Networks in Bayesian Optimisation
AU - Samuel Younger
AU - Tinkle Chugh
AU - George De Ath
PY - 2026
DA - 2026/10/27
PB - arXiv
DO - 10.48550/arxiv.2610.37737
UR - https://arxiv.org/abs/2610.37737
AB - Prior-Data Fitted Networks (PFNs) replace the hyperparameter refitting of Gaussian Process (GP) surrogates in Bayesian optimisation with a single forward pass, at the cost of a prior that is fixed at meta-training time. We show empirically that a PFN's predictive log-likelihood falls increasingly behind that of a refitted GP as the objective function's lengthscale grows beyond the mode of the PFN's meta-training prior, and that this deficit carries over to optimisation performance. To correct the mismatch without retraining the network, we introduce Anisotropic Trust Region (ATR) warping. Per-dimension lengthscales are estimated by kernel target alignment, and an affine rescaling of the input space maps the objective onto the lengthscale the PFN expects, with a trust region around the anchor determining which observations are retained, leaving the PFN's 𝒪(d · N^2) inference cost unchanged. On functions drawn from a Matérn-3/2 kernel, ATR warping improves optimisation performance of the PFN at both extremes of the lengthscale range. Our implementation is publicly available at https://github.com/youngers2006/Trust-Region-Warping-For-PFNs.
ER -