{
  "abstract": "BackgroundRisk-of-bias (ROB) assessments represent an integral component of systematic reviews. However, this task is often highly repetitive, time-consuming, and may lack inter-rater consistency. Large language models (LLMs) offer opportunities for automation in systematic reviews, which may expedite and enhance the quality and consistency of research synthesis.\n\nMethodsUsing zero-shot prompting, we designed an LLM-based pipeline as a virtual mimic of a human reviewer for the Quality in Prognosis Studies (QUIPS) framework. Then, focusing on prognostic research in a single discipline (neurology), we applied this pipeline to articles included in previously published systematic reviews. We studied inter-rater agreement between both (1) the LLM and the original human ROB assessments and (2) between original human ROB assessments.\n\nResults298 articles from 15 reviews across three domains (epilepsy, traumatic brain injury, stroke) were included. We demonstrate the feasibility of a tailored, prompt-engineered LLM pipeline for automating ROB assessments with the QUIPS tool. While LLM-human agreement was limited (Cohens weighted kappa; = 0.22, 95% CI, 0.12 - 0.33), our data tentatively suggest, based on a small sample (n=5), that it may not be inferior to human-human agreement (Cohens weighted kappa; = -0.25, 95% CI, -1.04 - 0.54). Wilcoxon signed-rank tests were statistically significant (p < 0.05) across four bias domains and for the overall risk scores, and rank-biserial correlations demonstrated human raters tendency to assign higher risk scores than LLM counterparts.\n\nConclusionsWith targeted methodological refinements - including standardization of QUIPS implementation and validation against expert ratings - automated ROB assessments may meaningfully reduce time and cost of systematic reviews of prognosis studies in neurology and beyond.",
  "abstract_shown": true,
  "affiliations": [
    "Department of Child Neurology, UMC Utrecht Brain Center, University Medical Center Utrecht and Utrecht University, Utrecht, The Netherlands, member of ERN EpiCA"
  ],
  "article_type": "PUBLISHAHEADOFPRINT",
  "arxiv": null,
  "author_total": 3,
  "authors": [
    {
      "name": "S. L. Kampman"
    },
    {
      "name": "K. P. Braun"
    },
    {
      "affs": [
        0
      ],
      "name": "W. M. Otte"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/amn8k5yr9fqp4.bib",
  "cited_by": 0,
  "doi": "10.64898/2026.09.09.26362643",
  "fulltext_url": "https://www.medrxiv.org/content/10.64898/2026.09.09.26362643v1.full",
  "id": "amn8k5yr9fqp4",
  "issns": [],
  "landing_url": "https://www.medrxiv.org/content/10.64898/2026.09.09.26362643v1",
  "language": null,
  "licence": "CC-BY",
  "licence_label": "CC BY",
  "licence_source": "article",
  "licence_url": "https://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "medrxiv",
      "neurology"
    ],
    [
      "openalex",
      "1804|Meta-analysis and systematic reviews"
    ]
  ],
  "openalex": "W7212443686",
  "pdf_url": "https://www.medrxiv.org/content/10.64898/2026.09.09.26362643v1.full.pdf",
  "pmcid": null,
  "pmid": null,
  "published": "2026-09-09",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/amn8k5yr9fqp4.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:18:30Z",
      "native_id": "10.64898/2026.09.09.26362643",
      "source": "medrxiv",
      "url": "https://www.medrxiv.org/content/10.64898/2026.09.09.26362643v1"
    }
  ],
  "subjects": [
    {
      "field": "medicine",
      "origin": "medrxiv:neurology",
      "subfield": "neurology"
    }
  ],
  "title": "A large language model for risk-of-bias assessment in systematic reviews of prognosis studies in clinical neurology",
  "title_text": "A large language model for risk-of-bias assessment in systematic reviews of prognosis studies in clinical neurology",
  "type": "preprint",
  "updated_at": "2026-10-30T12:18:30Z",
  "url": "https://research.draconitic.com/paper/amn8k5yr9fqp4",
  "venue": "medRxiv",
  "versions": []
}