Moving beyond stationarity: A sliding window z-score criterion for response time outlier exclusion
Abstract
The traditional approach of identifying response time (RT) outliers calculates exclusion thresholds across the entire sample of RTs, thereby neglecting any non-stationarity in RT such as practice-related RT changes. Consequently, the probability of a trial being excluded as outlier can be confounded with its position within the experimental session, e.g., as slow RTs tend to come more from the beginning of the experiment. Here, we demonstrate such undesirable behavior for a traditional exclusion approach based on a constant z-score (e.g., mean RT ± 2 × SD), for which outliers were more likely to come from the beginning of the experiment the more trials were available. Critically, this constant method also introduced bias into the analysis of mean effects, and more specifically for effect estimates at the beginning of the experiment. Such bias could be reduced by an exclusion method based on moving z-scores, which identifies outliers according to z-scores obtained by a sliding window, therefore accounting for global changes in RT. Identifying outliers according to a moving threshold thus more appropriately accounts for the characteristics of evolving RTs, moving beyond the stationarity assumption inherent in the traditional approaches. R code for implementing this procedure is provided.
Related
- Support Vector Machines for Classification, Regression, and Outlier Detection: A Tutorial
- Model-Agnostic Influential Outlier Detection for Mixed Effects and Multi-Level Models
- Robust Dual-Regularized Variable Selection under Outlier Contamination
- A unified approach to outlier identification for mixed-type data
- Local Law and Outlier Eigenvalues of Spiked Separable Covariance Matrices
BibTeX
@misc{berger2026moving,
title = {{Moving beyond stationarity: A sliding window z-score criterion for response time outlier exclusion}},
author = {Alexander Berger and Markus Kiefer},
howpublished = {PsyArXiv},
year = {2026},
month = oct,
doi = {10.31234/osf.io/ktbcu_v2},
url = {https://osf.io/preprints/psyarxiv/ktbcu_v2/},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Moving beyond stationarity: A sliding window z-score criterion for response time outlier exclusion
AU - Alexander Berger
AU - Markus Kiefer
PY - 2026
DA - 2026/10/20
PB - PsyArXiv
DO - 10.31234/osf.io/ktbcu_v2
UR - https://osf.io/preprints/psyarxiv/ktbcu_v2/
AB - The traditional approach of identifying response time (RT) outliers calculates exclusion thresholds across the entire sample of RTs, thereby neglecting any non-stationarity in RT such as practice-related RT changes. Consequently, the probability of a trial being excluded as outlier can be confounded with its position within the experimental session, e.g., as slow RTs tend to come more from the beginning of the experiment. Here, we demonstrate such undesirable behavior for a traditional exclusion approach based on a constant z-score (e.g., mean RT ± 2 × SD), for which outliers were more likely to come from the beginning of the experiment the more trials were available. Critically, this constant method also introduced bias into the analysis of mean effects, and more specifically for effect estimates at the beginning of the experiment. Such bias could be reduced by an exclusion method based on moving z-scores, which identifies outliers according to z-scores obtained by a sliding window, therefore accounting for global changes in RT. Identifying outliers according to a moving threshold thus more appropriately accounts for the characteristics of evolving RTs, moving beyond the stationarity assumption inherent in the traditional approaches. R code for implementing this procedure is provided.
ER -