TestBED: An Open-Source Toolkit for Bayesian Experimental Design
Abstract
Bayesian experimental design (BED) is a principled framework for intelligent data acquisition with applications across a wide range of scientific and industrial domains. However, the development and adoption of BED remains hindered by a lack of standardised software and benchmarks. Existing methods are typically implemented in bespoke codebases and evaluated on a small subset of tasks, making methods difficult to apply, compare, or extend. To address this, we present TestBED, a general-purpose, open-source toolkit for Bayesian experimental design. TestBED decomposes the BED process into modular, reusable components spanning tasks, objectives, training mechanisms, policy architectures, and evaluation procedures, together with scalable implementations of core primitives such as expected information gain estimators. We provide a wide range of these components, which can be freely composed, allowing existing methods to be reproduced within a common framework, while also enabling previously unexplored configurations and new methodologies to be easily implemented and evaluated. Using TestBED, we further conduct a large-scale benchmark of contemporary BED methods, including controlled studies of under-appreciated but surprisingly critical methodological choices, such as the importance of time embeddings in design policy architectures. By providing a modular, extensible software foundation and a reproducible benchmarking suite, we hope that TestBED will accelerate methodological research in BED and lower the barrier to its real-world application.
Related
- Power-posterior implicit sampling for efficient Bayesian experimental design: Application to contaminant source identification in groundwater flow
- FairBED: Incorporating Fairness into Bayesian Experimental Design
- Certified Bayesian optimal experimental design: from Expected Information Gain to Signal-to-Noise Ratio
- Risk-Aware Goal-Oriented Bayesian Optimal Experimental Design
- Transfer learning in bayesian optimization for aircraft design
BibTeX
@misc{kerrigan2026testbed,
title = {{TestBED: An Open-Source Toolkit for Bayesian Experimental Design}},
author = {Gavin Kerrigan and Marcel Hedman and Sahel Iqbal and Tom Rainforth},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.06633},
eprint = {2611.06633},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.06633},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - TestBED: An Open-Source Toolkit for Bayesian Experimental Design
AU - Gavin Kerrigan
AU - Marcel Hedman
AU - Sahel Iqbal
AU - Tom Rainforth
PY - 2026
DA - 2026/11/04
PB - arXiv
DO - 10.48550/arxiv.2611.06633
UR - https://arxiv.org/abs/2611.06633
AB - Bayesian experimental design (BED) is a principled framework for intelligent data acquisition with applications across a wide range of scientific and industrial domains. However, the development and adoption of BED remains hindered by a lack of standardised software and benchmarks. Existing methods are typically implemented in bespoke codebases and evaluated on a small subset of tasks, making methods difficult to apply, compare, or extend. To address this, we present TestBED, a general-purpose, open-source toolkit for Bayesian experimental design. TestBED decomposes the BED process into modular, reusable components spanning tasks, objectives, training mechanisms, policy architectures, and evaluation procedures, together with scalable implementations of core primitives such as expected information gain estimators. We provide a wide range of these components, which can be freely composed, allowing existing methods to be reproduced within a common framework, while also enabling previously unexplored configurations and new methodologies to be easily implemented and evaluated. Using TestBED, we further conduct a large-scale benchmark of contemporary BED methods, including controlled studies of under-appreciated but surprisingly critical methodological choices, such as the importance of time embeddings in design policy architectures. By providing a modular, extensible software foundation and a reproducible benchmarking suite, we hope that TestBED will accelerate methodological research in BED and lower the barrier to its real-world application.
ER -