Automation of literature screening for meta-analyses in the field of psychology: an evaluation of two text-mining approaches.
- 1Department of Cognition, Emotion, and Methods in Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria.
- 2Pfizer Pharma GmbH, Berlin, Germany.
- 3Department of Cognition, Emotion, and Methods in Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria. [email protected].
Abstract
Background
The process of consolidating research through meta-analyses is time-consuming and costly. Using text-mining approaches to (semi-)automate the step of screening and selecting articles eligible for inclusion might potentially save hours of workload and costs. A lack of evaluative research on the use of text-mining for literature screening prevents the widespread adoption of text-mining approaches among the psychology research community. This study explores the possibilities of applying text-mining approaches to facilitate or (semi-)automate screening articles for inclusion in psychology meta-analyses.Methods
The performances of two publicly accessible text-mining approaches were evaluated and compared against three recent meta-analyses from different subfields of psychology.Results
Averaged over 10 text-mining runs per meta-analysis, across the three meta-analyses the approaches achieved recall performances between 62.5% and 99.3%, specificities between 60.1% and 92.7%, and saved between 54.0% and 92.0% of screening workload. Higher recall was observed when more articles were screened manually in the text-mining process.Conclusions
These findings suggest that text-mining approaches can substantially reduce screening workload while maintaining high recall. These efficiency gains come with a trade-off: the more workload is saved, the lower the recall performance. Text-mining can therefore serve as a valuable tool for the initial identification of potentially eligible studies or as a secondary screening aid, but the risk of missing relevant studies should be mitigated by retaining an appropriate amount of manual screening.
Related
- Evaluation of the psychology of earthquake victims referred to plastic surgery service after crush injury: a cross-sectional study
- Twelve-Year Real-World Evaluation of a Regulated Guideline-Based Warfarin Dosing and Care Automation System
- Single-Stage Computed Tomography Angiography, Percutaneous Coronary Intervention, and Transcatheter Aortic Valve Replacement in an Octogenarian With Femoral Neck Fracture, Non-ST-Elevation Myocardial Infarction, and Severe Aortic Stenosis: A Case Report and Literature Review of Multidisciplinary Approaches
- A Specialized Model versus a Frontier Model in Emotional Support: Blind Evaluation by Psychology Professionals
- Hepatitis C screening strategies and linkage-to-care approaches in U.S. emergency departments: A systematic review and meta-analysis
BibTeX
@article{gloukhman2026automation,
title = {{Automation of literature screening for meta-analyses in the field of psychology: an evaluation of two text-mining approaches.}},
author = {Dennis Gloukhman and Caroline Lade and Ulrich S Tran},
journal = {Systematic reviews},
year = {2026},
month = aug,
doi = {10.1186/s13643-026-03275-x},
url = {https://europepmc.org/article/MED/42968199},
note = {CC BY},
}
RIS
TY - JOUR
TI - Automation of literature screening for meta-analyses in the field of psychology: an evaluation of two text-mining approaches.
AU - Dennis Gloukhman
AU - Caroline Lade
AU - Ulrich S Tran
PY - 2026
DA - 2026/08/05
JO - Systematic reviews
DO - 10.1186/s13643-026-03275-x
UR - https://europepmc.org/article/MED/42968199
AB - Background The process of consolidating research through meta-analyses is time-consuming and costly. Using text-mining approaches to (semi-)automate the step of screening and selecting articles eligible for inclusion might potentially save hours of workload and costs. A lack of evaluative research on the use of text-mining for literature screening prevents the widespread adoption of text-mining approaches among the psychology research community. This study explores the possibilities of applying text-mining approaches to facilitate or (semi-)automate screening articles for inclusion in psychology meta-analyses.Methods The performances of two publicly accessible text-mining approaches were evaluated and compared against three recent meta-analyses from different subfields of psychology.Results Averaged over 10 text-mining runs per meta-analysis, across the three meta-analyses the approaches achieved recall performances between 62.5% and 99.3%, specificities between 60.1% and 92.7%, and saved between 54.0% and 92.0% of screening workload. Higher recall was observed when more articles were screened manually in the text-mining process.Conclusions These findings suggest that text-mining approaches can substantially reduce screening workload while maintaining high recall. These efficiency gains come with a trade-off: the more workload is saved, the lower the recall performance. Text-mining can therefore serve as a valuable tool for the initial identification of potentially eligible studies or as a secondary screening aid, but the risk of missing relevant studies should be mitigated by retaining an appropriate amount of manual screening.
SN - 2046-4053
ER -