Post Hoc Confidence Bounds On False Positives Using Learned Templates
Abstract
This paper establishes the theoretical validity of the use of adaptive templates in the context of simultaneous control of the False Discovery Proportion (FDP). Previous work, Blain et al. (2022), has shown that combining data-driven templates with the inference framework of Blanchard et al. (2020) improves the quality of the resulting post-hoc bounds, however the validity guarantees provided relied on datasplitting. In this work we demonstrate that using data-driven templates, learnt on the same dataset via resampling, provides asymptotically valid inference. We prove that these templates converge as the sample size increases, and use this to establish asymptotic control of the joint error rate and provide valid simultaneous lower bounds for the FDP as a consequence. We show that using a single round of bootstraps to learn the templates is sufficient for validity, allowing for a speed up in the inference procedure. We provide experiments validating the approach, showing that using the same dataset to learn the templates and perform inference is valid in practice and increases power relative to data-splitting.
Related
- Modeling Word Counts: Common Practice Risks False Positives
- On strongly coherent confidence envelopes for the false discoveries
- Approximation Property of Dropout Neural Networks: Sobolev Rates and Confidence Bounds
- Variance-adaptive Chernoff bounds: nonasymptotically valid, asymptotically sharp confidence sets for unbounded means
- Colored Turán densities and finite directed templates
BibTeX
@misc{davenport2026post,
title = {{Post Hoc Confidence Bounds On False Positives Using Learned Templates}},
author = {Samuel Davenport and Bertrand Thirion and Pierre Neuvial},
howpublished = {HAL},
year = {2026},
month = jun,
url = {https://hal.science/hal-05778262v1},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Post Hoc Confidence Bounds On False Positives Using Learned Templates
AU - Samuel Davenport
AU - Bertrand Thirion
AU - Pierre Neuvial
PY - 2026
DA - 2026/06/30
PB - HAL
UR - https://hal.science/hal-05778262v1
AB - This paper establishes the theoretical validity of the use of adaptive templates in the context of simultaneous control of the False Discovery Proportion (FDP). Previous work, Blain et al. (2022), has shown that combining data-driven templates with the inference framework of Blanchard et al. (2020) improves the quality of the resulting post-hoc bounds, however the validity guarantees provided relied on datasplitting. In this work we demonstrate that using data-driven templates, learnt on the same dataset via resampling, provides asymptotically valid inference. We prove that these templates converge as the sample size increases, and use this to establish asymptotic control of the joint error rate and provide valid simultaneous lower bounds for the FDP as a consequence. We show that using a single round of bootstraps to learn the templates is sufficient for validity, allowing for a speed up in the inference procedure. We provide experiments validating the approach, showing that using the same dataset to learn the templates and perform inference is valid in practice and increases power relative to data-splitting.
ER -