GLLaucoMed: A Secure Large Language Model-Powered Agentic Workflow for Automated Medication Extraction from Free-Text Glaucoma Clinical Notes.
4 affiliations
- 1Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida.
- 2Department of Ophthalmology and Visual Sciences, Escola Paulista de Medicina Universidade Federal de São Paulo, São Paulo, São Paulo, Brazil.
- 3Department of Ophthalmology, Olive View - UCLA Medical Center, Sylmar, California.
- 4Miller School of Medicine, University of Miami, Miami, Florida.
Abstract
Purpose
To evaluate the efficacy of large language models (LLMs) in extracting medication-related information from glaucoma clinical notes in the electronic health record.Design
Cross-sectional.Subjects
One thousand two hundred fifty subjects in the Bascom Palmer Ophthalmic Repository.Methods
Extracted clinical notes from glaucoma-related encounters between 2014 and 2024 were randomly sampled and labeled by 2 glaucoma specialists, with a third serving as an adjudicator. Graders were asked to label current topical medications (CTM), proposed changes to topical medications (ΔTM), current oral medications (COM), and proposed changes to oral medications (ΔOM) in a structured fashion. The data set was split into development (10%), validation (10%), and test (80%) sets stratified by clinician. Development and validation sets were used to engineer and refine prompts, and the held-out test set was used for model assessment. Five LLMs (Claude Opus 4.6, DeepSeek-V3.2, GPT 5.2, Grok 4.1, and Qwen3.6-35B-A3B) were accessed via Microsoft Azure AI Foundry within a Health Insurance Portability and Accountability Act-compliant environment. Intergrader agreement was assessed with Gwet AC1. Large language model performance was initially assessed in a binary fashion with F1 scores, and the degree of text match among positive cases was evaluated using exact match accuracy and Jaccard Index (JI).Main outcome measures
F1 score, exact match accuracy, JI.Results
Gwet AC1 for intergrader agreement was 0.799, 0.888, 0.985, and 0.988 for CTM, ΔTM, COM, and ΔOM, respectively. F1 scores for CTM were 0.985, 0.971, 0.978, 0.968, and 0.970 for Claude, DeepSeek, GPT, Grok, and Qwen, respectively; for ΔTM: 0.905, 0.826, 0.897, 0.842, and 0.865, respectively; for COM: 0.923, 0.887, 0.899, 0.906, and 0.894, respectively; for ΔOM: 0.958, 0.815, 0.937, 0.835, and 0.940, respectively. Among positive cases, range of exact match accuracies for CTM (n = 1354) was 0.730-0.882, and range of JIs was 0.809-0.918. For ΔTM (n = 404), exact match accuracy range was 0.619 to 0.780, and JI range was 0.668-0.827. For COM (n = 47), exact match accuracy range was 0.766 to 0.872, and JI range was 0.765 to 0.870. For ΔOM (n = 25), exact match accuracy range was 0.583 to 0.920, and JI range was 0.583 to 0.922.Conclusions
The GLLaucoMed pipeline demonstrated high performance in extracting and standardizing medication data from unstructured clinical notes, including both current medications and proposed changes. Claude and GPT exhibited the strongest performance.Financial disclosures
Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
Related
- An automated operon refactoring workflow for heterologous expression of non-ribosomal peptide biosynthetic gene clusters
- Effect of guideline-assisted prompting on large language model performance in medication-related osteonecrosis of the jaw–related endodontic decision-making
- Usability and workflow implications of an electronic health record-integrated tool for automated phenotyping: Formative evaluation of Pheno+
- Large Language Models in the Radiology Workflow: Current Applications, Challenges, and Future Perspectives.
- Bern Sleep-Wake Registry: temporal extraction of comorbidities from unstructured clinical text using a large language model
BibTeX
@article{solages2026gllaucomed,
title = {{GLLaucoMed: A Secure Large Language Model-Powered Agentic Workflow for Automated Medication Extraction from Free-Text Glaucoma Clinical Notes.}},
author = {Nicholas Solages and Rafael Scherer and Gustavo A Samico and Naomi E Gutkind and Janet Kang and Felipe A Medeiros and Swarup S Swaminathan},
journal = {Ophthalmology science},
year = {2026},
month = aug,
doi = {10.1016/j.xops.2026.101366},
url = {https://europepmc.org/article/MED/42927148},
note = {CC BY-NC-ND},
}
RIS
TY - JOUR
TI - GLLaucoMed: A Secure Large Language Model-Powered Agentic Workflow for Automated Medication Extraction from Free-Text Glaucoma Clinical Notes.
AU - Nicholas Solages
AU - Rafael Scherer
AU - Gustavo A Samico
AU - Naomi E Gutkind
AU - Janet Kang
AU - Felipe A Medeiros
AU - Swarup S Swaminathan
PY - 2026
DA - 2026/08/17
JO - Ophthalmology science
DO - 10.1016/j.xops.2026.101366
UR - https://europepmc.org/article/MED/42927148
AB - Purpose To evaluate the efficacy of large language models (LLMs) in extracting medication-related information from glaucoma clinical notes in the electronic health record.Design Cross-sectional.Subjects One thousand two hundred fifty subjects in the Bascom Palmer Ophthalmic Repository.Methods Extracted clinical notes from glaucoma-related encounters between 2014 and 2024 were randomly sampled and labeled by 2 glaucoma specialists, with a third serving as an adjudicator. Graders were asked to label current topical medications (CTM), proposed changes to topical medications (ΔTM), current oral medications (COM), and proposed changes to oral medications (ΔOM) in a structured fashion. The data set was split into development (10%), validation (10%), and test (80%) sets stratified by clinician. Development and validation sets were used to engineer and refine prompts, and the held-out test set was used for model assessment. Five LLMs (Claude Opus 4.6, DeepSeek-V3.2, GPT 5.2, Grok 4.1, and Qwen3.6-35B-A3B) were accessed via Microsoft Azure AI Foundry within a Health Insurance Portability and Accountability Act-compliant environment. Intergrader agreement was assessed with Gwet AC1. Large language model performance was initially assessed in a binary fashion with F1 scores, and the degree of text match among positive cases was evaluated using exact match accuracy and Jaccard Index (JI).Main outcome measures F1 score, exact match accuracy, JI.Results Gwet AC1 for intergrader agreement was 0.799, 0.888, 0.985, and 0.988 for CTM, ΔTM, COM, and ΔOM, respectively. F1 scores for CTM were 0.985, 0.971, 0.978, 0.968, and 0.970 for Claude, DeepSeek, GPT, Grok, and Qwen, respectively; for ΔTM: 0.905, 0.826, 0.897, 0.842, and 0.865, respectively; for COM: 0.923, 0.887, 0.899, 0.906, and 0.894, respectively; for ΔOM: 0.958, 0.815, 0.937, 0.835, and 0.940, respectively. Among positive cases, range of exact match accuracies for CTM (n = 1354) was 0.730-0.882, and range of JIs was 0.809-0.918. For ΔTM (n = 404), exact match accuracy range was 0.619 to 0.780, and JI range was 0.668-0.827. For COM (n = 47), exact match accuracy range was 0.766 to 0.872, and JI range was 0.765 to 0.870. For ΔOM (n = 25), exact match accuracy range was 0.583 to 0.920, and JI range was 0.583 to 0.922.Conclusions The GLLaucoMed pipeline demonstrated high performance in extracting and standardizing medication data from unstructured clinical notes, including both current medications and proposed changes. Claude and GPT exhibited the strongest performance.Financial disclosures Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
SN - 2666-9145
ER -