Leveraging prompt-driven generative AI for systematic reviews in digital psychiatry: A stage-matched comparative proof-of-concept for healthcare researchers and clinicians

@article{parkington2026leveraging,
  title = {{Leveraging prompt-driven generative AI for systematic reviews in digital psychiatry: A stage-matched comparative proof-of-concept for healthcare researchers and clinicians}},
  author = {Karisa Parkington and Jithin T. Joseph and Katie Musleh and Marianne Rouleau-Tang and Annabelle Persaud and Alice Rueda and Bazen Gashaw Teferra and Huda F. Al-Shamali and Wanying Mao and Fatemeh Gholamali Nezhad and Lisa Burback and Yanbo Zhang and Rakesh Jetly and Eric Vermetten and Candice Monson and Sri Krishnan and Richard J. Zeifman and Andrew Greenshaw and Wendy Lou and Venkat Bhat},
  journal = {PLOS Digital Health},
  year = {2026},
  month = sep,
  doi = {10.1371/journal.pdig.0001666},
  url = {https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001666},
  note = {CC BY 4.0},
}

TY  - JOUR
TI  - Leveraging prompt-driven generative AI for systematic reviews in digital psychiatry: A stage-matched comparative proof-of-concept for healthcare researchers and clinicians
AU  - Karisa Parkington
AU  - Jithin T. Joseph
AU  - Katie Musleh
AU  - Marianne Rouleau-Tang
AU  - Annabelle Persaud
AU  - Alice Rueda
AU  - Bazen Gashaw Teferra
AU  - Huda F. Al-Shamali
AU  - Wanying Mao
AU  - Fatemeh Gholamali Nezhad
AU  - Lisa Burback
AU  - Yanbo Zhang
AU  - Rakesh Jetly
AU  - Eric Vermetten
AU  - Candice Monson
AU  - Sri Krishnan
AU  - Richard J. Zeifman
AU  - Andrew Greenshaw
AU  - Wendy Lou
AU  - Venkat Bhat
PY  - 2026
DA  - 2026/09/10
JO  - PLOS Digital Health
DO  - 10.1371/journal.pdig.0001666
UR  - https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001666
AB  - The scale and pace of evidence generation in digital psychiatry increasingly exceed the capacity of traditional systematic literature review (SLR) methods. Large language models (LLMs) are gaining traction in evidence synthesis, yet limited guidance exists for integrating generative AI into transparent, reproducible SLR workflows. To develop and evaluate a prompt-driven, modular decision-making framework for adaptable SLR workflows in digital psychiatry, and to compare stage-matched performance relative to a consensus-based human reference process. We conducted a mixed-methods, stage-matched comparative evaluation of three GPT-5 mode variants (Auto, Agent, and Deep Research) against a consensus-based human-led SLR workflow using a registered digital psychiatry review (PROSPERO CRD42025648122) as a case example. Nine SLR tasks were implemented within the ChatGPT interface using structured RISEN prompts: preliminary searches; research question, eligibility criteria, and search strategy development; article screening; data extraction; article summarization; critical appraisals; and descriptive results synthesis. Performance was evaluated using task-specific assessments of accuracy, sensitivity, specificity, inter-rater agreement, reporting compliance, hallucination monitoring, and workflow feasibility. All GPT-5 mode variants were feasible across SLR stages, although performance varied by task and mode. No fabricated study-level content was identified during structured hallucination auditing. Auto mode performed best for structured rule-based tasks requiring efficiency and implementation-ready outputs (e.g., eligibility criteria). Agent mode excelled in conceptual integration and interpretive tasks (e.g., research question generation, critical appraisals, descriptive synthesis). Deep Research mode most closely approximated human reasoning for higher-order synthesis tasks, performing best in full-text article screening, article summarization, and narrative synthesis. Prompt-driven LLM workflows are feasible and semi-efficacious for selected SLR tasks in digital psychiatry when deployed within a modular, human-in-the-loop decision-support framework. Findings support task-specific deployment of an adaptable modular framework, enabling healthcare researchers and clinicians to modify LLM workflows according to methodological appropriateness, confidence in performance, and institutional review practices. Author summary: Systematic literature reviews (SLRs) are essential for summarizing scientific evidence, but they are often time-intensive and require substantial human effort. Recent advances in artificial intelligence (AI), including large language models (LLMs) such as ChatGPT, have raised interest in whether these tools could support or accelerate parts of the review process. However, little is known about how well different AI configurations perform across the many stages of a systematic review. In this proof-of-concept study, we evaluated an adaptable, step-by-step workflow showing how healthcare researchers and clinicians could practically use different versions of GPT-5 to assist with nine major stages of an SLR in psychiatry. Rather than testing a fully automated system, we examined how AI might support specific parts of the process while keeping human researchers involved in reviewing and verifying outputs. We found that performance varied depending on both the task and GPT-5 mode used. Some modes performed well for structured tasks such as screening and summarizing findings, whereas all modes demonstrated limitations in extracting numerical data and critically evaluating evidence quality. Overall, our findings suggest that AI may meaningfully support evidence synthesis for end-users with limited AI background when carefully supervised, but currently works best as a flexible decision-support tool rather than a replacement for expert judgement.
SN  - 2767-3170
ER  -