The Benjamini-Hochberg procedure can fail to control the FDR for correlated two-sided Gaussian tests.
- 1Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, PA 19104.
Abstract
The Benjamini-Hochberg (BH) procedure is the standard method for controlling the false discovery rate (FDR) in large-scale studies involving testing multiple hypotheses. Whether the BH procedure controls the FDR for every correlation structure among two-sided Gaussian tests has been a long-standing question, and a positive answer has been believed. Here, we settle the question negatively, by providing examples of Gaussian factor models for which the BH method fails to control the FDR at the nominal level. We prove mathematically that the violation holds, and also provide simulation results which are consistent with the theory. The level of violation can be characterized as mild but nonnegligible. In one of our examples, empirical FDR levels of ≈0.0106 and ≈0.053 are achieved for nominal FDR levels 0.01 and 0.05, respectively. Larger violations are also possible. The violation appears most prominently for large numbers (thousands) of tests; though we also present an example with a moderate number of 85 tests. We also briefly discuss the implications of our findings.
Related
- How Much Can Gaussian Dependence Inflate the Benjamini-Hochberg Procedure's FDR?
- Sharp Asymptotic Minimaxity of the Gavrilov-Benjamini-Sarkar Step-Down Testing Procedure in Sparse Gaussian Sequence Models
- Asymptotic Bayes Optimality Under Sparsity of the Gavrilov-Benjamini-Sarkar Step-Down Testing Procedure
- Correlated Gaussian Moments Determine Polynomials up to Orthogonal Equivalence
- Scaling limit for the pinning model in correlated Gaussian environment beyond the L2-regime
BibTeX
@article{dobriban2026benjaminihochberg,
title = {{The Benjamini-Hochberg procedure can fail to control the FDR for correlated two-sided Gaussian tests.}},
author = {Edgar Dobriban},
journal = {Proceedings of the National Academy of Sciences of the United States of America},
year = {2026},
month = oct,
doi = {10.1073/pnas.2626267123},
url = {https://europepmc.org/article/MED/42832672},
note = {CC BY-NC-ND},
}
RIS
TY - JOUR
TI - The Benjamini-Hochberg procedure can fail to control the FDR for correlated two-sided Gaussian tests.
AU - Edgar Dobriban
PY - 2026
DA - 2026/10/05
JO - Proceedings of the National Academy of Sciences of the United States of America
DO - 10.1073/pnas.2626267123
UR - https://europepmc.org/article/MED/42832672
AB - The Benjamini-Hochberg (BH) procedure is the standard method for controlling the false discovery rate (FDR) in large-scale studies involving testing multiple hypotheses. Whether the BH procedure controls the FDR for every correlation structure among two-sided Gaussian tests has been a long-standing question, and a positive answer has been believed. Here, we settle the question negatively, by providing examples of Gaussian factor models for which the BH method fails to control the FDR at the nominal level. We prove mathematically that the violation holds, and also provide simulation results which are consistent with the theory. The level of violation can be characterized as mild but nonnegligible. In one of our examples, empirical FDR levels of ≈0.0106 and ≈0.053 are achieved for nominal FDR levels 0.01 and 0.05, respectively. Larger violations are also possible. The violation appears most prominently for large numbers (thousands) of tests; though we also present an example with a moderate number of 85 tests. We also briefly discuss the implications of our findings.
SN - 0027-8424
SN - 1091-6490
ER -