Skip to content
DraconiticResearch
Search Sources API
197,768 studies31 sourcesUpdated 11 min ago
PreprintarXiv licencearXiv

Do I Even Need a Multilevel Model? Negligible Effect Significance Testing for Intraclass Correlation Coefficients

  • Dayeon Lee
  • Gregory R. Hancock
Open at sourcePDFDOICite

Abstract

In applied research, the decision to utilize multilevel modeling (MLM) is commonly guided by comparing a point estimate of the unconditional intraclass correlation coefficient (ICC) against some recommended threshold (e.g., 0.05). This naive approach, however, fails to account for sampling uncertainty and provides no formal inferential justification for a decision. The current work introduces negligible effect significance testing (NEST; or equivalence testing) for the ICC, proposing a framework that performs this test using a pivotal quantity of the ICC based on the F statistic. Furthermore, we offer guidance to help researchers define "negligible" that utilizes the design effect to characterize a tolerable level of variance inflation. R code illustrating the proposed procedure is provided.

Versions

Related

  1. Multilevel Monte Carlo for Polytopic DG Discretisations with Spatially Represented Random Coefficients
    Veronika Chronholm, Matt Evans, Emmanuil H. Georgoulis +1
    arXivPreprintCC BY 4.0Full textNumerical analysis & scientific computingarXiv
  2. Matrix-Aware Proper Scoring Rules and Significance Testing for Correlation and Covariance Forecasts in Python
    Vinh Nguyen
    arXivPreprintCC BY 4.0Full textStatisticsarXiv
  3. Intracluster correlation coefficients for neonatal trials: an analysis of national population level clinical data.
    S Jawad, A T Prevost, C Gale +1
    TrialsPeer-reviewedCC BYFull textStatisticsEurope PMCDOAJ
  4. An analog of Quillen-Venkov for eventually negligible classes and radicality for finite irreducible Coxeter groups
    Constantin Cedillo Vayson de Pradenne, Matthew Gherman, Enoch Guo +1
    arXivPreprintCC BY 4.0Full textAlgebra & number theoryarXiv
  5. Negligible in Size, Significant in Effect: On Scale Vectors in Large Language Models
    Mingze Wang, Shuchen Zhu, Yuxin Fang +3
    arXivPreprintarXiv licenceFull textStatisticsarXiv

Cite

BibTeXRIS
BibTeX
@misc{lee2026even,
  title = {{Do I Even Need a Multilevel Model? Negligible Effect Significance Testing for Intraclass Correlation Coefficients}},
  author = {Dayeon Lee and Gregory R. Hancock},
  howpublished = {arXiv},
  year = {2026},
  month = aug,
  doi = {10.48550/arxiv.2608.02535},
  eprint = {2608.02535},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.02535},
}
RIS
TY  - UNPB
TI  - Do I Even Need a Multilevel Model? Negligible Effect Significance Testing for Intraclass Correlation Coefficients
AU  - Dayeon Lee
AU  - Gregory R. Hancock
PY  - 2026
DA  - 2026/08/03
PB  - arXiv
DO  - 10.48550/arxiv.2608.02535
UR  - https://arxiv.org/abs/2608.02535
AB  - In applied research, the decision to utilize multilevel modeling (MLM) is commonly guided by comparing a point estimate of the unconditional intraclass correlation coefficient (ICC) against some recommended threshold (e.g., 0.05). This naive approach, however, fails to account for sampling uncertainty and provides no formal inferential justification for a decision. The current work introduces negligible effect significance testing (NEST; or equivalence testing) for the ICC, proposing a framework that performs this test using a pivotal quantity of the ICC based on the F statistic. Furthermore, we offer guidance to help researchers define "negligible" that utilizes the design effect to characterize a tolerable level of variance inflation. R code illustrating the proposed procedure is provided.
ER  -