The effect of manuscript characteristics on quality assessment of scientific manuscripts in psychology
- 1Department Methodology and Statistics, Tilburg University, the Netherlands
- 2Department of Sociology, Utrecht University, the Netherlands
Abstract
We report a vignette study and a survey to investigate which study characteristics influence quality ratings psychology academics give of articles submitted for publication, and psychology academics and students to theses written by psychology students. In the vignette study, 800 respondents evaluated the quality of an abstract (7-point scale) of studies with small (N = 54) or large sample sizes (N = 540), showing statistically significant or non-significant results, and containing statistical reporting errors or no errors. In the subsequent survey, the same participants rated the importance of 29 manuscript characteristics related to a study’s theory, design, conduct, data analyses, and presentation for assessing either the quality of a manuscript or its publishability (article) or grade (thesis). Results showed that quality ratings were affected by sample sizes but not by statistical significance or the presence of statistical reporting errors in the rated research vignette. Furthermore, in the survey academics and students provided highly similar ratings of the importance of different aspects relevant to quality assessment of articles and theses. These results suggest that quality criteria for scientific manuscripts are already adopted by students and are similar for submitted manuscripts and theses. Further research is needed to examine which study characteristics researchers consider most important when assessing a manuscript’s quality in peer review.
Related
- A Call to Action: Open and Inclusive Assessment Measures in Developmental Psychology
- Associations between social network characteristics and daily social experiences in later life: evidence from ecological momentary assessment
- Assessment of resting-state prefrontal haemodynamic fluctuation characteristics in male university students who have received curling training using fNIRS technology
- Wearable inertial sensor assessment of kinematic characteristics during unilateral jump landings in elite volleyball players with patellar tendinopathy: a cross-sectional study
- Do ‘tortured phrases’ exist in the psychology literature, and are there psychology-specific ‘tortured phrases’?
BibTeX
@article{augusteijn2026effect,
title = {{The effect of manuscript characteristics on quality assessment of scientific manuscripts in psychology}},
author = {Hilde E.M. Augusteijn and Jelte M. Wicherts and Klaas Sijtsma and Marcel A.L.M. van Assen},
journal = {PLOS One},
year = {2026},
month = sep,
doi = {10.1371/journal.pone.0355938},
url = {https://doi.org/10.1371/journal.pone.0355938},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - The effect of manuscript characteristics on quality assessment of scientific manuscripts in psychology
AU - Hilde E.M. Augusteijn
AU - Jelte M. Wicherts
AU - Klaas Sijtsma
AU - Marcel A.L.M. van Assen
PY - 2026
DA - 2026/09/15
JO - PLOS One
DO - 10.1371/journal.pone.0355938
UR - https://doi.org/10.1371/journal.pone.0355938
AB - We report a vignette study and a survey to investigate which study characteristics influence quality ratings psychology academics give of articles submitted for publication, and psychology academics and students to theses written by psychology students. In the vignette study, 800 respondents evaluated the quality of an abstract (7-point scale) of studies with small (N = 54) or large sample sizes (N = 540), showing statistically significant or non-significant results, and containing statistical reporting errors or no errors. In the subsequent survey, the same participants rated the importance of 29 manuscript characteristics related to a study’s theory, design, conduct, data analyses, and presentation for assessing either the quality of a manuscript or its publishability (article) or grade (thesis). Results showed that quality ratings were affected by sample sizes but not by statistical significance or the presence of statistical reporting errors in the rated research vignette. Furthermore, in the survey academics and students provided highly similar ratings of the importance of different aspects relevant to quality assessment of articles and theses. These results suggest that quality criteria for scientific manuscripts are already adopted by students and are similar for submitted manuscripts and theses. Further research is needed to examine which study characteristics researchers consider most important when assessing a manuscript’s quality in peer review.
SN - 1932-6203
ER -