Dangers of Bayesian analyses and how to address them
Abstract
In light of the US FDA announcement supporting the use of Bayesian methods in clinical trials, we present a nontechnical review of problems with Bayesian analyses and methods to address them. Our focus is on the well-known sensitivities of Bayesian results to biases embedded in their prior distributions. This vulnerability calls for special safety features, including detailed justification of informative priors based on empirical-research literature rather than on singular expert opinions. Crucially, the information contributed by priors should be realistic in light of actual background data rather than merely judged so by available experts. We illustrate our recommendations with using a textbook clinical-trial example which covered these points. We then use the example describe basic diagnostic procedures and safety recommendations for Bayesian analyses and their presentation. These include reference analyses which exclude informative priors, and justification for informative priors based on empirical-research literature rather than on philosophical arguments. We also critique some common priors that violate our realism requirements. One strategy to aid judgments about priors is to translate them into data to be added to the study data. The source of these prior data should be examined with the same critical attitude as the source of study data. We illustrate some simple summary methods for comparing prior and data information, including prior predictions of data, variance ratios, and their translation to effective sample sizes (ESS). Finally, we argue that Bayesian analyses should be treated as extensions or supplements to conventional analyses, rather than replace them, for that treatment puts prior construction and justification at the fore of the transition from frequentist to Bayesian analyses in teaching and practice.
Related
- A Subjective Bayesian Approach to Address Unchecked Assumptions in a Causal Analysis
- Optimally Emulating Smooth Functions for Bayesian Analyses
- Risk of a statistical blind spot: the dangers of data shortages
- One ring to rule them all: syzygy maximization for Argyres-Douglas
- Observable functions of rational ODE models and how to find them
BibTeX
@misc{greenland2026dangers,
title = {{Dangers of Bayesian analyses and how to address them}},
author = {Sander Greenland and Jason Oke},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2609.14082},
eprint = {2609.14082},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.14082},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Dangers of Bayesian analyses and how to address them
AU - Sander Greenland
AU - Jason Oke
PY - 2026
DA - 2026/10/12
PB - arXiv
DO - 10.48550/arxiv.2609.14082
UR - https://arxiv.org/abs/2609.14082
AB - In light of the US FDA announcement supporting the use of Bayesian methods in clinical trials, we present a nontechnical review of problems with Bayesian analyses and methods to address them. Our focus is on the well-known sensitivities of Bayesian results to biases embedded in their prior distributions. This vulnerability calls for special safety features, including detailed justification of informative priors based on empirical-research literature rather than on singular expert opinions. Crucially, the information contributed by priors should be realistic in light of actual background data rather than merely judged so by available experts. We illustrate our recommendations with using a textbook clinical-trial example which covered these points. We then use the example describe basic diagnostic procedures and safety recommendations for Bayesian analyses and their presentation. These include reference analyses which exclude informative priors, and justification for informative priors based on empirical-research literature rather than on philosophical arguments. We also critique some common priors that violate our realism requirements. One strategy to aid judgments about priors is to translate them into data to be added to the study data. The source of these prior data should be examined with the same critical attitude as the source of study data. We illustrate some simple summary methods for comparing prior and data information, including prior predictions of data, variance ratios, and their translation to effective sample sizes (ESS). Finally, we argue that Bayesian analyses should be treated as extensions or supplements to conventional analyses, rather than replace them, for that treatment puts prior construction and justification at the fore of the transition from frequentist to Bayesian analyses in teaching and practice.
ER -