Axioms for validity: e-values, p-values and data-dependent levels
Abstract
The emerging literature on hypothesis testing with data-dependent significance levels relies on a particular extension of the Type-I error. Existing arguments for this extension are heuristic. We show it is uniquely characterized by three axioms: law-invariance, agreement with classical testing on constant levels, and the preservation of (in)validity under independent selection of the level. We find that three analogous axioms characterize the e-value as a multi-decision generalization of a test in a decision-theoretic framework. We recover the p-value by dropping the invalidity half of the independent selection axiom, showing that e-values correspond to those p-values for which a data-independent choice between two invalid p-values cannot produce a valid p-value. Our main technical contribution is to show how e-values emerge in every decision problem with a notion of validity that satisfies law invariance, the independent selection axiom, and an axiom stating that a procedure that selects an invalid procedure with arbitrarily high probability must itself be invalid. In an application to post-hoc decision-making, we justify expectation-based error rates in multiple testing and recover the 'e-closure principle' as a special case.
Related
- Conditional validity of prediction-powered inference for spatially dependent finite populations
- Initialization-dependent BFGS trial rates for tilted absolute values
- Asymptotic Theory for Combining Dependent p-Values for Global Hypothesis Testing
- Forcing axioms and the structure theory of Aronszajn trees
- Extension Theorems in Measure and Integration Theory as Completeness Axioms
BibTeX
@misc{koning2026axioms,
title = {{Axioms for validity: e-values, p-values and data-dependent levels}},
author = {Nick W. Koning},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.28429},
eprint = {2605.28429},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.28429},
}
RIS
TY - UNPB
TI - Axioms for validity: e-values, p-values and data-dependent levels
AU - Nick W. Koning
PY - 2026
DA - 2026/10/19
PB - arXiv
DO - 10.48550/arxiv.2605.28429
UR - https://arxiv.org/abs/2605.28429
AB - The emerging literature on hypothesis testing with data-dependent significance levels relies on a particular extension of the Type-I error. Existing arguments for this extension are heuristic. We show it is uniquely characterized by three axioms: law-invariance, agreement with classical testing on constant levels, and the preservation of (in)validity under independent selection of the level. We find that three analogous axioms characterize the e-value as a multi-decision generalization of a test in a decision-theoretic framework. We recover the p-value by dropping the invalidity half of the independent selection axiom, showing that e-values correspond to those p-values for which a data-independent choice between two invalid p-values cannot produce a valid p-value. Our main technical contribution is to show how e-values emerge in every decision problem with a notion of validity that satisfies law invariance, the independent selection axiom, and an axiom stating that a procedure that selects an invalid procedure with arbitrarily high probability must itself be invalid. In an application to post-hoc decision-making, we justify expectation-based error rates in multiple testing and recover the 'e-closure principle' as a special case.
ER -