Coverage is not enough: Frequentist tests of simulation-based inference for primordial non-Gaussianity
Abstract
(Abridged) Simulation-based inference (SBI) has emerged as a powerful framework for extracting cosmological information from complex, non-linear data where analytical likelihoods are unavailable. Its reliability is commonly assessed using coverage-based diagnostics under the prior predictive distribution, which probe calibration only in an averaged sense and do not constrain posterior behavior at fixed parameter value, the regime relevant for practical inference. We investigate these limitations in the context of primordial non-Gaussianity, parameterized by fNL, using simulations of the dark matter halo field. We compare SBI based on contrastive neural ratio estimation (CNRE) with likelihood-based inference (LBI) using the power spectrum, bispectrum, and wavelet scattering transform (WST) coefficients across 1000 realizations. SBI and LBI agree well on posterior means and skewness, while the variance agrees on average but shows weaker realization-by-realization consistency. Larger differences arise in the kurtosis, indicating discrepancies in the posterior tails. These effects are already present for the power spectrum - where the Gaussian likelihood assumed in LBI is best justified - and are most pronounced for the combined power spectrum and bispectrum, where SBI posteriors are often underconfident and can yield weaker constraints than either statistic individually, despite passing coverage tests. WST coefficients further tighten constraints on fNL, even when restricted to large scales. Our results highlight both the potential of higher-order statistics and the need for validation strategies that probe the posterior shape beyond standard coverage diagnostics.
Versions
Related
- Euclid: Field-level inference of primordial non-Gaussianity and cosmic initial conditions
- Large primordial non-Gaussianity from transient turns in Higgs-R2 inflation
- Initial clustering of primordial black holes: A general formulation for arbitrary local non-Gaussianity
- Probing the local primordial non-Gaussianity with clustering of FRB dispersion measure and galaxies
- Predicting Primordial Non-Gaussianity Measurements in the CSST Spectroscopic Survey
BibTeX
@misc{alokda2026coverage,
title = {{Coverage is not enough: Frequentist tests of simulation-based inference for primordial non-Gaussianity}},
author = {Toka Alokda and Cristiano Porciani and Alexander Eggemeier},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.00980},
eprint = {2605.00980},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.00980},
}
RIS
TY - UNPB
TI - Coverage is not enough: Frequentist tests of simulation-based inference for primordial non-Gaussianity
AU - Toka Alokda
AU - Cristiano Porciani
AU - Alexander Eggemeier
PY - 2026
DA - 2026/10/07
PB - arXiv
DO - 10.48550/arxiv.2605.00980
UR - https://arxiv.org/abs/2605.00980
AB - (Abridged) Simulation-based inference (SBI) has emerged as a powerful framework for extracting cosmological information from complex, non-linear data where analytical likelihoods are unavailable. Its reliability is commonly assessed using coverage-based diagnostics under the prior predictive distribution, which probe calibration only in an averaged sense and do not constrain posterior behavior at fixed parameter value, the regime relevant for practical inference. We investigate these limitations in the context of primordial non-Gaussianity, parameterized by f_NL, using simulations of the dark matter halo field. We compare SBI based on contrastive neural ratio estimation (CNRE) with likelihood-based inference (LBI) using the power spectrum, bispectrum, and wavelet scattering transform (WST) coefficients across 1000 realizations. SBI and LBI agree well on posterior means and skewness, while the variance agrees on average but shows weaker realization-by-realization consistency. Larger differences arise in the kurtosis, indicating discrepancies in the posterior tails. These effects are already present for the power spectrum - where the Gaussian likelihood assumed in LBI is best justified - and are most pronounced for the combined power spectrum and bispectrum, where SBI posteriors are often underconfident and can yield weaker constraints than either statistic individually, despite passing coverage tests. WST coefficients further tighten constraints on f_NL, even when restricted to large scales. Our results highlight both the potential of higher-order statistics and the need for validation strategies that probe the posterior shape beyond standard coverage diagnostics.
ER -