Minimum Specification Perturbation: Robustness as Distance-to-Falsification in Causal Inference
Abstract
Empirical causal claims depend on many analyst decisions, from selecting covariates to choosing estimators. Existing robustness tools summarize how results vary across these choices, but, to the best of our knowledge, do not answer: \textbf{How many analyst decisions must change to reach a specification, which is a set of choices, whose confidence interval (CI) contains zero?} We introduce \emph{Minimum Specification Perturbation (MSP)}, the smallest number of changes. MSP is small under the null, grows with effect strength and captures distance-to-falsification information that dispersion-based summaries cannot report; when making decisions under weak effects, an MSP-based rule yields lower false-positive rates than dispersion-based rules. We show that Fragility Index and MSP measure orthogonal vulnerabilities: fragility to influential observations need not imply fragility to specification choices. On the LaLonde benchmark, MSP = 1 implies that one decision change makes the CI contain zero. We further provide exact permutation calibration under randomization and characterize computation, showing tractable cases under additive structure and NP-hardness in general.
Related
- Exact Selective Inference for Spectral Clustering via Piecewise-Constant Graph Perturbation
- Causal Inference in Metric Spaces via Distance Profiles
- Adaptive Experimentation Can Hide Model Error:Detection Limits and Falsification Budget
- Bi-balanced shifts and variable specification
- Thermodynamic formalism for non-compact systems with expansivity and specification
BibTeX
@misc{dang2026minimum,
title = {{Minimum Specification Perturbation: Robustness as Distance-to-Falsification in Causal Inference}},
author = {Hoang Dang and Luan Pham and Minh Nguyen},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2605.01579},
eprint = {2605.01579},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.01579},
}
RIS
TY - UNPB
TI - Minimum Specification Perturbation: Robustness as Distance-to-Falsification in Causal Inference
AU - Hoang Dang
AU - Luan Pham
AU - Minh Nguyen
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2605.01579
UR - https://arxiv.org/abs/2605.01579
AB - Empirical causal claims depend on many analyst decisions, from selecting covariates to choosing estimators. Existing robustness tools summarize how results vary across these choices, but, to the best of our knowledge, do not answer: \textbf{How many analyst decisions must change to reach a specification, which is a set of choices, whose confidence interval (CI) contains zero?} We introduce \emph{Minimum Specification Perturbation (MSP)}, the smallest number of changes. MSP is small under the null, grows with effect strength and captures distance-to-falsification information that dispersion-based summaries cannot report; when making decisions under weak effects, an MSP-based rule yields lower false-positive rates than dispersion-based rules. We show that Fragility Index and MSP measure orthogonal vulnerabilities: fragility to influential observations need not imply fragility to specification choices. On the LaLonde benchmark, MSP = 1 implies that one decision change makes the CI contain zero. We further provide exact permutation calibration under randomization and characterize computation, showing tractable cases under additive structure and NP-hardness in general.
ER -