Bayesian Superiority in On/Off analysis
Abstract
We present a detailed comparison of Bayesian criteria with three non-informative priors - flat, Jeffreys, and scale-invariant - for testing a signal against an unknown background and compare them with the classical frequentist Li-Ma approach in the On/Off problem. We perform Monte Carlo simulations for various background levels and evaluate the Li-Ma and Bayesian criteria by their Type I error rates. We then simulate a nonzero signal and compare the criteria in terms of Type II error rates. We find that both the Li-Ma criterion and the Bayesian criterion with the Jeffreys prior control Type I error well, excluding the case of small background b < 0.5; the Bayesian criterion with the Jeffreys prior is statistically more powerful than the Li-Ma criterion for b ≲ 40.
Related
- Bayesian and frequentist perspectives agree on dynamical dark energy
- LACHESIS: Robust stellar parameters through Bayesian model averaging of isochrone grids
- Toward Bayesian photometric cosmic chronometers: Application to VIPERS
- Solar spicule population properties inferred from IRIS Mg II k off-limb spectra using likelihood-free Bayesian inference
- An improved standardization framework for Type II-P supernovae using a hierarchical Bayesian approach
BibTeX
@misc{babenov2026bayesian,
title = {{Bayesian Superiority in On/Off analysis}},
author = {Gleb Babenov and Petr Satunin},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.11683},
eprint = {2609.11683},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.11683},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Bayesian Superiority in On/Off analysis
AU - Gleb Babenov
AU - Petr Satunin
PY - 2026
DA - 2026/09/17
PB - arXiv
DO - 10.48550/arxiv.2609.11683
UR - https://arxiv.org/abs/2609.11683
AB - We present a detailed comparison of Bayesian criteria with three non-informative priors - flat, Jeffreys, and scale-invariant - for testing a signal against an unknown background and compare them with the classical frequentist Li-Ma approach in the On/Off problem. We perform Monte Carlo simulations for various background levels and evaluate the Li-Ma and Bayesian criteria by their Type I error rates. We then simulate a nonzero signal and compare the criteria in terms of Type II error rates. We find that both the Li-Ma criterion and the Bayesian criterion with the Jeffreys prior control Type I error well, excluding the case of small background b < 0.5; the Bayesian criterion with the Jeffreys prior is statistically more powerful than the Li-Ma criterion for b ≲ 40.
ER -