Simulation-based cutoffs for conditional item fit in Rasch models: iterations, multiplicity correction, and decision stability
Abstract
Simulation-based cutoff values for conditional item fit can be estimated by parametric bootstrapping and reported as an interval of a chosen width. Johansson (2025) recommended this over rule-of-thumb critical values and gave iteration guidance based on detection rates and conditioned on sample size, without an account of the error rate that the interval width controls. This paper re-examines iterations and width against the familywise error rate (FWER), and compares FWER and false discovery rate control. Seven simulation studies vary item count and the targeting of misfitting items, with both dichotomous and polytomous conditions.
The interval width is shown to be the error rate parameter of the procedure, setting the FWER jointly with the number of items. Bootstrap p-values with a Westfall-Young FWER correction target that error rate directly, detect more misfit than a width-matched interval, and are calibrated from 400 iterations, with fewer false positives than Benjamini-Hochberg. Outfit was consistently less powerful than infit, confirming earlier dichotomous findings and extending them to polytomous items.
Updated recommendations include flagging items on the Westfall-Young FWER corrected p-value, using at least 400 iterations and 1000 to 2000 for a final analysis, and reporting the interval as a descriptive band at width .95. The described methods are available in the R package easyRasch2 and the jamovi module easyRasch2jmv.
Related
- Conditional-prevalence correction for liability-scale analyses of covariate-adjusted binary GWAS
- Relational Multiplicity in Group Synchrony: Properties, Cognitive Demands, and Methodological Framework
- Tricuspid valve leaflet multiplicity and Doppler-derived right-heart hemodynamic trajectories after surgical repair.
- Conditional Myeloid-Specific Inhibition of UBE2N Hinders YUMM1.7 Growth
- Conditional survival and dynamic prognostic nomogram for histologically defined anaplastic ganglioglioma: a SEER-based analysis
BibTeX
@misc{johansson2026simulationbased,
title = {{Simulation-based cutoffs for conditional item fit in Rasch models: iterations, multiplicity correction, and decision stability}},
author = {Magnus Johansson},
howpublished = {PsyArXiv},
year = {2026},
month = oct,
doi = {10.31234/osf.io/7pqz4_v3},
url = {https://osf.io/preprints/psyarxiv/7pqz4_v3/},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Simulation-based cutoffs for conditional item fit in Rasch models: iterations, multiplicity correction, and decision stability
AU - Magnus Johansson
PY - 2026
DA - 2026/10/30
PB - PsyArXiv
DO - 10.31234/osf.io/7pqz4_v3
UR - https://osf.io/preprints/psyarxiv/7pqz4_v3/
AB - Simulation-based cutoff values for conditional item fit can be estimated by parametric bootstrapping and reported as an interval of a chosen width. Johansson (2025) recommended this over rule-of-thumb critical values and gave iteration guidance based on detection rates and conditioned on sample size, without an account of the error rate that the interval width controls. This paper re-examines iterations and width against the familywise error rate (FWER), and compares FWER and false discovery rate control. Seven simulation studies vary item count and the targeting of misfitting items, with both dichotomous and polytomous conditions. The interval width is shown to be the error rate parameter of the procedure, setting the FWER jointly with the number of items. Bootstrap p-values with a Westfall-Young FWER correction target that error rate directly, detect more misfit than a width-matched interval, and are calibrated from 400 iterations, with fewer false positives than Benjamini-Hochberg. Outfit was consistently less powerful than infit, confirming earlier dichotomous findings and extending them to polytomous items. Updated recommendations include flagging items on the Westfall-Young FWER corrected p-value, using at least 400 iterations and 1000 to 2000 for a final analysis, and reporting the interval as a descriptive band at width .95. The described methods are available in the R package easyRasch2 and the jamovi module easyRasch2jmv.
ER -