{
  "abstract": "Large language models (LLMs) are increasingly used for qualitative analysis in substance use research, yet their performance relative to human coders remains underexplored. This study compares ChatGPT-4.0 with human coders in performing qualitative coding tasks using NIH grant abstracts as a test case, focusing on the identification and description of research innovations. Using a sample of NIH HEAL Initiative grant abstracts related to opioid overdose prevention, a total of 125 abstracts were independently coded by ChatGPT and humans to generate innovation descriptions, which were then evaluated by both human raters and ChatGPT for depth/detail and relevance/completeness using 5-point Likert scales. Identical instructions were used across all coding and evaluation stages. ChatGPT-generated descriptions were consistently rated higher than human-generated descriptions on both dimensions. Human evaluators rated ChatGPT outputs at an average of 4.47 for both depth/detail and relevance/completeness, compared to 3.33 and 3.24 for human outputs, respectively (F(1,176)=133.9, p < 0.001). These findings suggest that LLMs, when carefully prompted, can enhance the efficiency and quality of qualitative research evaluation.",
  "abstract_shown": true,
  "affiliations": [
    "Department of Population & Community Health, College of Public Health, University of North Texas Health, Fort Worth, Texas, United States of America",
    "Texas College of Osteopathic Medicine, University of North Texas Health, Fort Worth, Texas, United States of America",
    "College of Science & Engineering, Texas Christian University, Fort Worth, Texas, United States of America",
    "Friends Research Institute, Baltimore, Maryland, United States of America",
    "George Mason University, Schar School of Policy and Government, Fairfax, Virginia, United States of America",
    "University of Wisconsin Center for Health Enhancement Systems Studies, Department of Industrial & Systems Engineering, Madison, Wisconsin, United States of America"
  ],
  "article_type": "Research Article",
  "arxiv": null,
  "author_total": 9,
  "authors": [
    {
      "affs": [
        0
      ],
      "name": "Sarah A. Alkhatib"
    },
    {
      "affs": [
        1
      ],
      "name": "Nadiya Jiwa"
    },
    {
      "affs": [
        2
      ],
      "name": "Dallin Judd"
    },
    {
      "affs": [
        3
      ],
      "name": "Justin M. Luningham"
    },
    {
      "affs": [
        4
      ],
      "name": "Ginnie Sawyer-Morris"
    },
    {
      "affs": [
        5
      ],
      "name": "Merve Ulukaya"
    },
    {
      "name": "Todd Molfenter"
    },
    {
      "name": "Faye S. Taxman"
    },
    {
      "name": "Scott T. Walters"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/6jqbwxq7cv3pg.bib",
  "cited_by": 0,
  "doi": "10.1371/journal.pone.0343857",
  "fulltext_url": null,
  "id": "6jqbwxq7cv3pg",
  "issns": [
    "1932-6203"
  ],
  "landing_url": "https://doi.org/10.1371/journal.pone.0343857",
  "language": null,
  "licence": "CC-BY-4.0",
  "licence_label": "CC BY 4.0",
  "licence_source": "journal",
  "licence_url": "https://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "openalex",
      "3300|Computational and Text Analysis Methods"
    ],
    [
      "plos",
      "/Biology and life sciences/Neuroscience/Cognitive science/Cognitive psychology/Language"
    ],
    [
      "plos",
      "/Biology and life sciences/Neuroscience/Cognitive science/Cognitive psychology/Reasoning"
    ],
    [
      "plos",
      "/Biology and life sciences/Psychology/Behavior/Human performance"
    ],
    [
      "plos",
      "/Biology and life sciences/Psychology/Cognitive psychology/Language"
    ],
    [
      "plos",
      "/Biology and life sciences/Psychology/Cognitive psychology/Reasoning"
    ],
    [
      "plos",
      "/Computer and information sciences/Artificial intelligence"
    ],
    [
      "plos",
      "/Medicine and health sciences/Pain management/Analgesics/Opioids"
    ],
    [
      "plos",
      "/Medicine and health sciences/Pharmacology/Drugs/Analgesics/Opioids"
    ],
    [
      "plos",
      "/Medicine and health sciences/Pharmacology/Drugs/Opioids"
    ],
    [
      "plos",
      "/Physical sciences/Mathematics/Statistics/Statistical methods/Analysis of variance"
    ],
    [
      "plos",
      "/Research and analysis methods/Mathematical and statistical techniques/Statistical methods/Analysis of variance"
    ],
    [
      "plos",
      "/Research and analysis methods/Research assessment/Systematic reviews"
    ],
    [
      "plos",
      "/Research and analysis methods/Research design/Qualitative studies"
    ],
    [
      "plos",
      "/Social sciences/Psychology/Behavior/Human performance"
    ],
    [
      "plos",
      "/Social sciences/Psychology/Cognitive psychology/Language"
    ],
    [
      "plos",
      "/Social sciences/Psychology/Cognitive psychology/Reasoning"
    ]
  ],
  "openalex": "W7220915755",
  "pdf_url": "https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0343857&type=printable",
  "pmcid": null,
  "pmid": null,
  "published": "2026-10-08",
  "publisher": "Public Library of Science",
  "ris": "https://research.draconitic.com/paper/6jqbwxq7cv3pg.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:17:30Z",
      "native_id": "10.1371/journal.pone.0343857",
      "source": "plos",
      "url": "https://doi.org/10.1371/journal.pone.0343857"
    }
  ],
  "subjects": [
    {
      "field": "biology",
      "origin": "plos:neuroscience",
      "subfield": "neuroscience"
    },
    {
      "field": "mathematics",
      "origin": "plos:statistical methods",
      "subfield": "statistics"
    },
    {
      "field": "medicine",
      "origin": "plos:pharmacology",
      "subfield": "pharmacology-toxicology"
    }
  ],
  "title": "AI versus human coding of NIH grant abstracts",
  "title_text": "AI versus human coding of NIH grant abstracts",
  "type": "peer-reviewed",
  "updated_at": "2026-10-30T12:17:30Z",
  "url": "https://research.draconitic.com/paper/6jqbwxq7cv3pg",
  "venue": "PLOS One",
  "versions": []
}