Bayes Estimators with Performance Comparable to Empirical Bayes Estimators and Improved Local Robustness
Abstract
Bayes estimation has been extensively studied and widely used in statistics, decision theory, signal processing, machine learning, and system identification. Among its variants, empirical Bayes (EB) estimation has attracted considerable attention due to its favorable estimation performance and computational tractability. However, the direct plug-in dependence of an EB estimator on hyperparameters can make it locally sensitive to hyper-parameter perturbations. This paper considers the linear regression model and focuses on the EB estimator by employing the marginal maximum likelihood hyper-parameter estimator. For conciseness, this estimator is simply referred to as the EB estimator. Given a family of EB weighting functions, a generalized Bayes estimator is constructed with the same excess mean squared error (XMSE) as the corresponding EB estimator. Here, the XMSE is a second-order asymptotic measure of the mean squared error difference between the estimator of interest and the maximum likelihood estimator. Furthermore, the EB estimator is shown to be at most firstorder sensitive to hyper-parameter perturbations, whereas the constructed Bayes estimator is at most second-order sensitive, making it locally more robust. The computational complexities of these two estimators are also analyzed. In some cases, the constructed Bayes estimator can be computationally comparable to, or more efficient than, the EB estimator. These theoretical results are further supported by numerical simulations.
Related
- infomeasure-rs: A High-Performance Rust Library for Information-Theoretic Measures and Estimators
- Eigenvector distributions and optimal shrinkage estimators for large covariance and precision matrices
- Optimal Two‐Phase Sampling Designs for Generalized Raking Estimators With Multiple Parameters of Interest
- UMVUE-Type Estimators under Bregman Losses
- Difference-based variance estimators with repeated measurements
BibTeX
@misc{ju2026bayes,
title = {{Bayes Estimators with Performance Comparable to Empirical Bayes Estimators and Improved Local Robustness}},
author = {Yue Ju and Ying Wang and Jiabao He and Bo Wahlberg and Håkan Hjalmarsson},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.06696},
eprint = {2609.06696},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.06696},
}
RIS
TY - UNPB
TI - Bayes Estimators with Performance Comparable to Empirical Bayes Estimators and Improved Local Robustness
AU - Yue Ju
AU - Ying Wang
AU - Jiabao He
AU - Bo Wahlberg
AU - Håkan Hjalmarsson
PY - 2026
DA - 2026/09/06
PB - arXiv
DO - 10.48550/arxiv.2609.06696
UR - https://arxiv.org/abs/2609.06696
AB - Bayes estimation has been extensively studied and widely used in statistics, decision theory, signal processing, machine learning, and system identification. Among its variants, empirical Bayes (EB) estimation has attracted considerable attention due to its favorable estimation performance and computational tractability. However, the direct plug-in dependence of an EB estimator on hyperparameters can make it locally sensitive to hyper-parameter perturbations. This paper considers the linear regression model and focuses on the EB estimator by employing the marginal maximum likelihood hyper-parameter estimator. For conciseness, this estimator is simply referred to as the EB estimator. Given a family of EB weighting functions, a generalized Bayes estimator is constructed with the same excess mean squared error (XMSE) as the corresponding EB estimator. Here, the XMSE is a second-order asymptotic measure of the mean squared error difference between the estimator of interest and the maximum likelihood estimator. Furthermore, the EB estimator is shown to be at most firstorder sensitive to hyper-parameter perturbations, whereas the constructed Bayes estimator is at most second-order sensitive, making it locally more robust. The computational complexities of these two estimators are also analyzed. In some cases, the constructed Bayes estimator can be computationally comparable to, or more efficient than, the EB estimator. These theoretical results are further supported by numerical simulations.
ER -