{
  "abstract": "Background: The potential of large language models (LLMs) to automate and support pharmacoepidemiologic study design is an emerging area of interest, yet their reliability remains insufficiently characterized. General-purpose LLMs often display inaccuracies, while the comparative performance of specialized biomedical LLMs in this domain remains unknown.\n\nMethods: This study evaluated general-purpose LLMs (GPT-4o and DeepSeek-R1) versus biomedically fine-tuned LLMs (QuantFactory/Bio-Medical-Llama-3-8B-GGUF and Irathernotsay/qwen2-1.5B-medical_qa-Finetune) using 46 protocols (2018–2024) from the HMA-EMA Catalogue and Sentinel System. Performance was assessed across relevance, logic of justification, and ontology-code agreement across multiple coding systems using Least-to-Most (LTM) and Active Prompting strategies.\n\nResults: GPT-4o and DeepSeek-R1 paired with LTM prompting achieved the highest relevance and logic of justification scores, with GPT-4o-LTM reaching a median relevance score of 4 in 8 of 9 questions for HMA-EMA protocols. Biomedical LLMs showed lower relevance overall and frequently generated insufficient justification. All LLMs demonstrated limited proficiency in ontology-code mapping.\n\nConclusion: Off-the-shelf general-purpose LLMs currently offered more reliable support for pharmacoepidemiologic study design than the smaller biomedical LLMs evaluated. In the model-controlled comparison (GPT-4o with Least-to-Most versus Active prompting), prompt strategy did not significantly affect performance (paired p = 0.93), and prompt comparisons are therefore reported descriptively. Because the two model groups also differed in scale and instruction-tuning, this contrast should be interpreted as a comparison of readily deployable options rather than of biomedical specialization alone.\n\nSupplementary Information: The online version contains supplementary material available at https://doi.org/10.1007/s11095-026-04190-5.",
  "abstract_shown": true,
  "affiliations": [
    "Department of Drug Design and Pharmacology, University of Copenhagen, Jagtvej 160, 2100 Copenhagen, Denmark"
  ],
  "article_type": "research article",
  "arxiv": null,
  "author_total": 5,
  "authors": [
    {
      "affs": [
        0
      ],
      "name": "Xinyao Zhang"
    },
    {
      "affs": [
        0
      ],
      "name": "Nicole Sonne Heckmann"
    },
    {
      "affs": [
        0
      ],
      "name": "Manuela Del Castillo Suero"
    },
    {
      "affs": [
        0
      ],
      "name": "Francesco Paolo Speca"
    },
    {
      "affs": [
        0
      ],
      "name": "Maurizio Sessa",
      "orcid": "0000-0003-0874-4744"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/d7485aytzgzcd.bib",
  "cited_by": 0,
  "doi": "10.1007/s11095-026-04190-5",
  "fulltext_url": "https://pmc.ncbi.nlm.nih.gov/articles/PMC13736468/",
  "id": "d7485aytzgzcd",
  "issns": [
    "0724-8741",
    "1573-904X"
  ],
  "landing_url": "https://doi.org/10.1007/s11095-026-04190-5",
  "language": "en",
  "licence": "CC-BY-4.0",
  "licence_label": "CC BY 4.0",
  "licence_source": "article",
  "licence_url": "https://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "heading",
      "Original Research Article"
    ],
    [
      "openalex",
      "2718|Artificial Intelligence in Healthcare and Education"
    ]
  ],
  "openalex": "W7155055113",
  "pdf_url": "https://link.springer.com/content/pdf/10.1007/s11095-026-04190-5.pdf",
  "pmcid": "PMC13736468",
  "pmid": "42728483",
  "published": "2026-09-11",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/d7485aytzgzcd.ris",
  "sources": [
    {
      "first_seen": "2026-11-09T12:21:54Z",
      "native_id": "PMC13736468",
      "source": "pmc",
      "url": "https://doi.org/10.1007/s11095-026-04190-5"
    }
  ],
  "subjects": [
    {
      "field": "medicine",
      "origin": "openalex:2718|Artificial Intelligence in Healthcare and Education",
      "subfield": "epidemiology-public-health"
    }
  ],
  "title": "Employing General-Purpose and Biomedical Large Language Models with Advanced Prompt Engineering for Pharmacoepidemiologic Study Design",
  "title_text": "Employing General-Purpose and Biomedical Large Language Models with Advanced Prompt Engineering for Pharmacoepidemiologic Study Design",
  "type": "peer-reviewed",
  "updated_at": "2026-11-09T12:21:54Z",
  "url": "https://research.draconitic.com/paper/d7485aytzgzcd",
  "venue": "Pharmaceutical Research",
  "versions": []
}