Bayesian additive regression trees for evaluating treatment benefit predictors using observational data
Abstract
A treatment benefit predictor (TBP) is an algorithm that maps a patient's characteristics to their putative benefit from a given treatment, which can be used to inform treatment decisions. However, a TBP must be evaluated in the target population before being adopted for patient care. When only observational data are available, evaluating TBPs requires standard causal identification assumptions, as treatment assignment in such settings is not random. We obtain the posterior distributions of predictive performance measures to evaluate prespecified TBPs using observational data, by taking advantage of Bayesian additive regression trees (BART). We illustrate the evaluation of TBPs using selected measures and graphical visualizations: the concentration of benefit (Cb) index and the moderate calibration curve. Simulation studies of binary and continuous outcomes settings, including balanced and imbalanced treatment allocation in the binary setting, establish the validity of the proposed approach. In a case study, we use this approach to assess a TBP for systemic antibiotic therapy for patients with chronic obstructive pulmonary disease (COPD). We show that the constructed TBP does not in fact make calibrated predictions, because it both makes optimistic predictions of risks and exaggerates the risk reduction due to treatment. We conclude that flexible Bayesian approaches have the potential to assess TBPs, offering opportunities for flexible model specifications, adjustment for confounding, and uncertainty characterization.
Related
- OLS Testing with Predictors Scaled to Have Unit Sample Variance, and Similar Results for a Robust Regression Estimator
- A Stochastic EM Algorithm with Sampling-Importance Resampling for Missing Data in Regression with Nonlinear Predictors
- Penalized Cox Regression with Graph-Structured Distribution-Valued Predictors
- Bayesian Variable Selection for High-Dimensional Predictors with Missing Psychometric Outcomes
- Kumaraswamy-Teissier Distribution: Bayesian and Non-Bayesian Inference, Regression Modeling and Climate Application
BibTeX
@misc{xia2026bayesian,
title = {{Bayesian additive regression trees for evaluating treatment benefit predictors using observational data}},
author = {Yuan Xia and Mohsen Sadatsafavi and Jeenat Mehareen and Paul Gustafson},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.21097},
eprint = {2609.21097},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.21097},
}
RIS
TY - UNPB
TI - Bayesian additive regression trees for evaluating treatment benefit predictors using observational data
AU - Yuan Xia
AU - Mohsen Sadatsafavi
AU - Jeenat Mehareen
AU - Paul Gustafson
PY - 2026
DA - 2026/09/17
PB - arXiv
DO - 10.48550/arxiv.2609.21097
UR - https://arxiv.org/abs/2609.21097
AB - A treatment benefit predictor (TBP) is an algorithm that maps a patient's characteristics to their putative benefit from a given treatment, which can be used to inform treatment decisions. However, a TBP must be evaluated in the target population before being adopted for patient care. When only observational data are available, evaluating TBPs requires standard causal identification assumptions, as treatment assignment in such settings is not random. We obtain the posterior distributions of predictive performance measures to evaluate prespecified TBPs using observational data, by taking advantage of Bayesian additive regression trees (BART). We illustrate the evaluation of TBPs using selected measures and graphical visualizations: the concentration of benefit (C_b) index and the moderate calibration curve. Simulation studies of binary and continuous outcomes settings, including balanced and imbalanced treatment allocation in the binary setting, establish the validity of the proposed approach. In a case study, we use this approach to assess a TBP for systemic antibiotic therapy for patients with chronic obstructive pulmonary disease (COPD). We show that the constructed TBP does not in fact make calibrated predictions, because it both makes optimistic predictions of risks and exaggerates the risk reduction due to treatment. We conclude that flexible Bayesian approaches have the potential to assess TBPs, offering opportunities for flexible model specifications, adjustment for confounding, and uncertainty characterization.
ER -