MAPEVAScR: a four-phase framework integrating transparent rule based text mining, evidence gap mapping and a mandatory human validation gate for scoping reviews framework development and empirical validation in occupational health.
- 1Universidad Privada san Juan Bautista
Abstract
BackgroundAutomation is increasingly used to reduce the screening workload of evidence syntheses, but its error is real, variable and rarely reported. The 2025 joint position statement of Cochrane, Campbell, JBI and the Collaboration for Environmental Evidence mandates human oversight of artificial intelligence in evidence synthesis, yet -- as of today -- no international guideline requires quantitative measurement and reporting of automated-screening error against a human gold standard (1).
MethodsWe developed MAPEVA-ScR (MAPeo de EVidencia con VAlidacion humana), a four-phase framework for scoping reviews: (A) datalake construction with deterministic, traceable deduplication; (B) transparent, record-level auditable rule-based text mining with confidence tiers; (C) evidence gap mapping that feeds back human screening prioritisation; and (D) a mandatory human validation gate verifying 100% of the candidate corpus and issuing a quantitative screening error report (per-tier precision, confusion matrix, false-positive families, documented changes to conclusions). The framework was empirically validated on a scoping review of epilepsy and occupational fitness (2015-2025; PubMed, Scopus, Web of Science): a funnel of 2,008 [->] 1,435 [->] 1,114 [->] 230 records was reduced to 127 human-confirmed inclusions (present study).
ResultsThe rule-based filter achieved a global precision of 55.2% (Tier A 75.5%, Tier B 37.5%, Tier C 45.8%); human validation identified 103 false positives classifiable into seven reproducible error families, and 107 of 127 included records required human correction of automated thematic labels. Without the gate, the reviews central thematic conclusion would have been published reversed: automated labelling ranked Aptitude/employability third (12.6% of records), whereas after validation it ranked first (73.2%) (present study).
ConclusionsA mandatory human validation gate with quantitative error metrics closes a reporting gap that current guidelines leave open: in the present study it prevented publication of an inverted synthesis. We propose the screening error report as a minimum reporting standard for any evidence synthesis using automated screening.
Registration and protocolNot applicable (methods development and validation study reporting original empirical data; no health intervention outcomes).
AI use disclosureGenerative AI assisted with language editing under full human verification (2); the MAPEVA-ScR pipeline uses deterministic rule-based text mining, with no generative AI in screening decisions.
Related
- Development and validation of a fall risk prediction model integrating objective gait metrics for patients with cerebral small vessel disease: protocol for a two-phase study
- Development and preliminary validation of an AI acceptance scale for intercultural teaching and its cross-sectional associations with university teachers’ occupational wellbeing
- Interpretable Machine Learning Prognostic Model Integrating Inflammatory-Nutritional Biomarkers for Post-Prostatectomy Biochemical Recurrence: Development, Internal Validation, and Translational Target Exploration.
- Development and validation of a prognostic tool for antenatal depressive symptoms: Statistical analysis plan for a two-stage model integrating self-reported and biomarker measures
- Development and validation of a risk stratification model for mild cognitive impairment among older adults with possible sarcopenia integrating traditional Chinese medicine constitution: a multicenter cross-sectional study
BibTeX
@misc{levano2026mapevascr,
title = {{MAPEVAScR: a four-phase framework integrating transparent rule based text mining, evidence gap mapping and a mandatory human validation gate for scoping reviews framework development and empirical validation in occupational health.}},
author = {C. J. ERAS LEVANO},
howpublished = {medRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.08.26362521},
url = {https://www.medrxiv.org/content/10.64898/2026.09.08.26362521v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - MAPEVAScR: a four-phase framework integrating transparent rule based text mining, evidence gap mapping and a mandatory human validation gate for scoping reviews framework development and empirical validation in occupational health.
AU - C. J. ERAS LEVANO
PY - 2026
DA - 2026/09/08
PB - medRxiv
DO - 10.64898/2026.09.08.26362521
UR - https://www.medrxiv.org/content/10.64898/2026.09.08.26362521v1
AB - BackgroundAutomation is increasingly used to reduce the screening workload of evidence syntheses, but its error is real, variable and rarely reported. The 2025 joint position statement of Cochrane, Campbell, JBI and the Collaboration for Environmental Evidence mandates human oversight of artificial intelligence in evidence synthesis, yet -- as of today -- no international guideline requires quantitative measurement and reporting of automated-screening error against a human gold standard (1). MethodsWe developed MAPEVA-ScR (MAPeo de EVidencia con VAlidacion humana), a four-phase framework for scoping reviews: (A) datalake construction with deterministic, traceable deduplication; (B) transparent, record-level auditable rule-based text mining with confidence tiers; (C) evidence gap mapping that feeds back human screening prioritisation; and (D) a mandatory human validation gate verifying 100% of the candidate corpus and issuing a quantitative screening error report (per-tier precision, confusion matrix, false-positive families, documented changes to conclusions). The framework was empirically validated on a scoping review of epilepsy and occupational fitness (2015-2025; PubMed, Scopus, Web of Science): a funnel of 2,008 [->] 1,435 [->] 1,114 [->] 230 records was reduced to 127 human-confirmed inclusions (present study). ResultsThe rule-based filter achieved a global precision of 55.2% (Tier A 75.5%, Tier B 37.5%, Tier C 45.8%); human validation identified 103 false positives classifiable into seven reproducible error families, and 107 of 127 included records required human correction of automated thematic labels. Without the gate, the reviews central thematic conclusion would have been published reversed: automated labelling ranked Aptitude/employability third (12.6% of records), whereas after validation it ranked first (73.2%) (present study). ConclusionsA mandatory human validation gate with quantitative error metrics closes a reporting gap that current guidelines leave open: in the present study it prevented publication of an inverted synthesis. We propose the screening error report as a minimum reporting standard for any evidence synthesis using automated screening. Registration and protocolNot applicable (methods development and validation study reporting original empirical data; no health intervention outcomes). AI use disclosureGenerative AI assisted with language editing under full human verification (2); the MAPEVA-ScR pipeline uses deterministic rule-based text mining, with no generative AI in screening decisions.
ER -