Skip to content
DraconiticResearch
Search Sources API
203,234 studies31 sourcesUpdated just now
PreprintarXiv licencearXiv

Estimating the True Effect Size Distribution with SIMEX

  • Zhaoqi Li
  • Daniel Ting
  • Ilya Gorbachev
  • Ehsan Emamjomeh-Zadeh
  • Houssam Nassif
Open at sourcePDFDOICite

Abstract

Large-scale online experimentation produces noisy effect estimates, which can overstate gains and complicate decisions about launches and testing policies. We propose a nonparametric method based on SIMulation-EXtrapolation (SIMEX) to estimate the latent distribution of true effects from estimated average treatment effects with known variances. The method evaluates quantiles after adding progressively more simulated measurement noise and extrapolates the resulting inverse cumulative distribution function to the zero-noise setting while enforcing monotonicity of the quantiles. In a synthetic example with normally distributed true effects and measurement error, the method recovers the underlying effect distribution and performs nearly as well as a parametric empirical Bayes normal means approach. This provides a flexible way to characterize effect-size distributions without fully specifying a parametric model.

Related

  1. Uniform Discrete Diffusion Models are Minimax Optimal for Estimating Distributions with Small Effective Support Size
    Dongsun Yoon, Saptarshi Chakraborty
    arXivPreprintarXiv licenceFull textStatisticsarXiv
  2. A Framework for Constructing, Estimating, and Validating the Bivariate Generalized Log-Gamma Distribution
    Arief Rachman Hakim, Purhadi, Jerry Dwi Trijoyo Purnomo
    MethodsXPeer-reviewedCC BY 4.0Full textStatisticsPMC
  3. Bayesian Sample Size Determination: Sampling Distribution Estimation or Exploration?
    Luke Hagar, Paul Gustafson
    arXivPreprintCC BY-NC-ND 4.0Full textStatisticsarXiv
  4. The Eaton--Moretó Conjecture Holds True for p-Solvable Groups
    Yanjun Liu, Lizhong Wang, Jiping Zhang +1
    arXivPreprintCC BY 4.0Full textAlgebra & number theoryarXiv
  5. A true Π01 sentence independence of PA
    Mohammad Golshani
    arXivPreprintarXiv licenceFull textLogic & foundationsarXiv

Cite

BibTeXRIS
BibTeX
@misc{li2026estimating,
  title = {{Estimating the True Effect Size Distribution with SIMEX}},
  author = {Zhaoqi Li and Daniel Ting and Ilya Gorbachev and Ehsan Emamjomeh-Zadeh and Houssam Nassif},
  howpublished = {arXiv},
  year = {2026},
  month = aug,
  doi = {10.48550/arxiv.2608.23612},
  eprint = {2608.23612},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.23612},
}
RIS
TY  - UNPB
TI  - Estimating the True Effect Size Distribution with SIMEX
AU  - Zhaoqi Li
AU  - Daniel Ting
AU  - Ilya Gorbachev
AU  - Ehsan Emamjomeh-Zadeh
AU  - Houssam Nassif
PY  - 2026
DA  - 2026/08/21
PB  - arXiv
DO  - 10.48550/arxiv.2608.23612
UR  - https://arxiv.org/abs/2608.23612
AB  - Large-scale online experimentation produces noisy effect estimates, which can overstate gains and complicate decisions about launches and testing policies. We propose a nonparametric method based on SIMulation-EXtrapolation (SIMEX) to estimate the latent distribution of true effects from estimated average treatment effects with known variances. The method evaluates quantiles after adding progressively more simulated measurement noise and extrapolates the resulting inverse cumulative distribution function to the zero-noise setting while enforcing monotonicity of the quantiles. In a synthetic example with normally distributed true effects and measurement error, the method recovers the underlying effect distribution and performs nearly as well as a parametric empirical Bayes normal means approach. This provides a flexible way to characterize effect-size distributions without fully specifying a parametric model.
ER  -