On the nineteenth-century origins of significance testing and p-hacking
Abstract
Although the names \emph{significance test}, \emph{p-value}, and \emph{confidence interval} came into use only in the 20th century, the methods they name were already used and abused in the 19th century. Knowledge of this earlier history can help us evaluate some of the ideas for improving statistical testing and estimation currently being discussed. This article recounts first the development of statistical testing and estimation after Laplace's discovery of the central limit theorem and then the subsequent transmission of these ideas into the English-language culture of mathematical statistics in the early 20th century. I argue that the earlier history casts doubt on the efficacy of many of the competing proposals for improving on significance tests and p-values and for forestalling abuses. Rather than further complicate the way we now teach statistics, we should leave aside most of the 20th-century embellishments and emphasize exploratory data analysis and the idea of testing probabilities by betting against them.
Related
- Do I Even Need a Multilevel Model? Negligible Effect Significance Testing for Intraclass Correlation Coefficients
- Matrix-Aware Proper Scoring Rules and Significance Testing for Correlation and Covariance Forecasts in Python
- How to post-train on a surrogate: Envelope sampling mitigates reward hacking
- Climate Sensitivity from 21st-Century Trends in Earth's Energy Imbalance
- Analog and hybrid computing in the 21st century
BibTeX
@misc{shafer2026nineteenthcentury,
title = {{On the nineteenth-century origins of significance testing and p-hacking}},
author = {Glenn Shafer},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.06233},
eprint = {2610.06233},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.06233},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - On the nineteenth-century origins of significance testing and p-hacking
AU - Glenn Shafer
PY - 2026
DA - 2026/10/05
PB - arXiv
DO - 10.48550/arxiv.2610.06233
UR - https://arxiv.org/abs/2610.06233
AB - Although the names \emph{significance test}, \emph{p-value}, and \emph{confidence interval} came into use only in the 20th century, the methods they name were already used and abused in the 19th century. Knowledge of this earlier history can help us evaluate some of the ideas for improving statistical testing and estimation currently being discussed. This article recounts first the development of statistical testing and estimation after Laplace's discovery of the central limit theorem and then the subsequent transmission of these ideas into the English-language culture of mathematical statistics in the early 20th century. I argue that the earlier history casts doubt on the efficacy of many of the competing proposals for improving on significance tests and p-values and for forestalling abuses. Rather than further complicate the way we now teach statistics, we should leave aside most of the 20th-century embellishments and emphasize exploratory data analysis and the idea of testing probabilities by betting against them.
ER -