Robust Estimation of Prior Parameters in Normal-Normal Empirical Bayes
Abstract
Many large-scale A/B testing systems leverage normal-normal empirical Bayes models to improve estimates of treatment impacts and launch decisions. We show that in these models maximum likelihood estimates of the variance of true impacts can be sensitive to a small number of extreme observations. We propose a bounded influence maximum likelihood estimator (BIMLE) that downweights such observations in proportion to their inconsistency with the model, reducing sensitivity while retaining the full sample in estimation. Absent extreme observations, BIMLE coincides with the standard MLE. In an application to A/B tests run at Amazon, BIMLE reduces instability in the estimated prior variance by over 50 percent while downweighting less than 0.1 percent of observations.
Related
- Robust estimation in generalized linear models based on the normal quantiles of the probability integral transformation
- Distributional Flexibility under Box-Cox Normality: A Theoretical and Empirical Evaluation of the Power-Normal Distribution
- CARB: A Covariate-Assessed Robust Borrowing Strategy with Literature-Informed Prior Weights for External Data
- Modelling nuisance parameters in linear models: Formal and empirical comparisons between the two-way Anova and Ancova
- Robust and Sparse Group Dynamic Causal Modeling via Student-t Parametric Empirical Bayes and Nonlocal Priors
BibTeX
@misc{burson2026robust,
title = {{Robust Estimation of Prior Parameters in Normal-Normal Empirical Bayes}},
author = {Manu Burson and Ryan Kessler},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.00326},
eprint = {2611.00326},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.00326},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Robust Estimation of Prior Parameters in Normal-Normal Empirical Bayes
AU - Manu Burson
AU - Ryan Kessler
PY - 2026
DA - 2026/11/01
PB - arXiv
DO - 10.48550/arxiv.2611.00326
UR - https://arxiv.org/abs/2611.00326
AB - Many large-scale A/B testing systems leverage normal-normal empirical Bayes models to improve estimates of treatment impacts and launch decisions. We show that in these models maximum likelihood estimates of the variance of true impacts can be sensitive to a small number of extreme observations. We propose a bounded influence maximum likelihood estimator (BIMLE) that downweights such observations in proportion to their inconsistency with the model, reducing sensitivity while retaining the full sample in estimation. Absent extreme observations, BIMLE coincides with the standard MLE. In an application to A/B tests run at Amazon, BIMLE reduces instability in the estimated prior variance by over 50 percent while downweighting less than 0.1 percent of observations.
ER -