When Can Ultra-Short Paired-Comparison Assessment Recover Trait Information? A Simulation and Empirical Illustration
Abstract
Ultra-short comparative assessments reduce respondent burden, but compression also reduces the information available for score recovery. This article evaluates ultra-short graded paired-comparison (GPC) assessment as a design-contingent information-recovery problem. Study 1 examined 324 Monte Carlo data-generating conditions varying construct saturation, stimulus validity, nuisance variance, pair construction, and test length, with three response formats and multiple recovery methods evaluated within each condition. Across 1,000 replications per condition, design factors dominated average format differences. GPC responses recovered benchmark information under favorable conditions, retained more information than binary paired choices, and deteriorated when target signal was weak, nuisance variance was systematic, or pair construction was contaminated. Reliability could increase while benchmark recovery declined. Study 2 applied this framework to three-pair image-GPC versions of the STAI-6 and SABAS. The STAI implementation showed strong benchmark recovery across constrained ordinal scoring, keyed and component scores, cross-validated prediction, and descriptive multitrait-multimethod checks. The SABAS implementation showed weak recovery across the same diagnostics. The findings support a conservative conclusion: ultra-short GPC assessment can work, but only when the construct, stimuli, pair construction, response format, length, and scoring method jointly preserve enough target-relevant information.
Related
- Social comparison information is not always prioritized: Temporal performance feedback dominates self-assessment in dysphoria
- Dual-informant clinical outcome assessment in the inclusive orchestra: measurement architecture for neurodevelopmental disorders prior to empirical evaluation
- Filling the information gap: a validated comparison of ChatGPT, Gemini, and Claude for penile cancer patient education
- Medical Decision‐Making Capacity in Early‐Stage Alzheimer Patients: Comparison With Healthy Controls and Benefit of Assistive Information
- Diagnostic pathways for gastroesophageal reflux cough: From empirical recognition to MII-pH-based multidimensional assessment
BibTeX
@misc{gallardopujol2026when,
title = {{When Can Ultra-Short Paired-Comparison Assessment Recover Trait Information? A Simulation and Empirical Illustration}},
author = {David Gallardo-Pujol and Laura Viñals-Vilà and Núria Arimany and Elisenda Tarrats Pons},
howpublished = {PsyArXiv},
year = {2026},
month = nov,
doi = {10.31234/osf.io/kjts2_v1},
url = {https://osf.io/preprints/psyarxiv/kjts2_v1/},
note = {CC0},
}
RIS
TY - UNPB
TI - When Can Ultra-Short Paired-Comparison Assessment Recover Trait Information? A Simulation and Empirical Illustration
AU - David Gallardo-Pujol
AU - Laura Viñals-Vilà
AU - Núria Arimany
AU - Elisenda Tarrats Pons
PY - 2026
DA - 2026/11/02
PB - PsyArXiv
DO - 10.31234/osf.io/kjts2_v1
UR - https://osf.io/preprints/psyarxiv/kjts2_v1/
AB - Ultra-short comparative assessments reduce respondent burden, but compression also reduces the information available for score recovery. This article evaluates ultra-short graded paired-comparison (GPC) assessment as a design-contingent information-recovery problem. Study 1 examined 324 Monte Carlo data-generating conditions varying construct saturation, stimulus validity, nuisance variance, pair construction, and test length, with three response formats and multiple recovery methods evaluated within each condition. Across 1,000 replications per condition, design factors dominated average format differences. GPC responses recovered benchmark information under favorable conditions, retained more information than binary paired choices, and deteriorated when target signal was weak, nuisance variance was systematic, or pair construction was contaminated. Reliability could increase while benchmark recovery declined. Study 2 applied this framework to three-pair image-GPC versions of the STAI-6 and SABAS. The STAI implementation showed strong benchmark recovery across constrained ordinal scoring, keyed and component scores, cross-validated prediction, and descriptive multitrait-multimethod checks. The SABAS implementation showed weak recovery across the same diagnostics. The findings support a conservative conclusion: ultra-short GPC assessment can work, but only when the construct, stimuli, pair construction, response format, length, and scoring method jointly preserve enough target-relevant information.
ER -