Identifying interstitial lung disease with large language models on clinical notes and claims
- 1Mass General Brigham Hospital
Abstract
Interstitial lung disease (ILD) is a potentially fatal toxicity of immune checkpoint inhibitors and antibody-drug conjugates. Drug-safety studies in healthcare databases identify ILD with claims-based algorithms that trade positive predictive value (PPV) against sensitivity. We assessed whether large language models (LLMs) improve on these algorithms, with and without electronic health record (EHR) notes, and whether these gains translate into measurable epidemiological benefits. In 309 oncology patients with ILD diagnosis codes at Mass General Brigham, adjudicated by two pulmonologists, we compared 12 open-access LLMs with 20 published claims-based algorithms. LLMs received claims, EHR notes, or both, without task-specific training. Measures were PPV and yield (proportion classified positive). We projected the attenuation of a drug-induced ILD odds ratio under non-differential outcome misclassification. LLMs extended the PPV-yield frontier of the published algorithms with all three inputs. Across the 36 model--input evaluations, LLM PPV exceeded that of the best algorithm without specialist-coding requirements (Algorithm 4) by 14.3 percentage points on average. With claims alone, Qwen3-8B achieved 89.8% PPV (95% CI 78.2-95.6) at 15.9% yield, versus 86.7% at 4.9% for the highest-PPV algorithm. With notes alone, Qwen3-32B achieved 78.9% PPV at 49.2% yield, versus 64.6% at 51.1% for Algorithm~4. With combined claims and notes, Qwen3-14B achieved 100.0% PPV at 24.3% yield. Under a simulated drug-safety scenario with a true odds ratio of 3.0, the projected observed odds ratio was 2.67 with combined inputs versus 1.07 with Algorithm 4, corresponding to a 46-fold difference in the estimated sample size required for 80% power (591 versus 27,200 patients). Zero-shot LLMs offer a higher-PPV alternative to rule-based claims phenotyping for ILD, on all three input types. Because the claims-only configuration uses the same claims data as existing algorithms, it could be deployed in administrative databases without linked notes, and may recover drug-safety signal otherwise lost to outcome misclassification.
Related
- Nomogram models integrating multiregion radiomics, quantitative computed tomography, and clinical indicators for distinguishing connective tissue disease-associated interstitial lung disease from idiopathic pulmonary fibrosis and improving the prediction of gender-age-physiology stage
- Do Calibrated-Probability Claims Hold Up in Clinical Diagnosis? A Pre-registered Evaluation of a Decision Model Against Frontier Language Models
- Mechanisms and Clinical Translational Prospects of the Microbiota–Gut–Lung Axis in Connective Tissue Disease-Associated Interstitial Lung Disease
- Clinical Characteristics and Outcomes of Anti-MDA5 Antibody-Associated Interstitial Lung Disease: A Single-Center Retrospective Cohort Study from an Integrated Advanced Lung Disease and Lung Transplant Program in the US
- [Analysis of Clinical Characteristics in 49 Patients of Lung Cancer Combined with Interstitial Lung Disease].
BibTeX
@misc{larghero2026identifying,
title = {{Identifying interstitial lung disease with large language models on clinical notes and claims}},
author = {G. Larghero and S. Schneeweiss and B. Gu and P. Anand and J. K. Lin and J. Yang},
howpublished = {medRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.30.26366942},
url = {https://www.medrxiv.org/content/10.64898/2026.10.30.26366942v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Identifying interstitial lung disease with large language models on clinical notes and claims
AU - G. Larghero
AU - S. Schneeweiss
AU - B. Gu
AU - P. Anand
AU - J. K. Lin
AU - J. Yang
PY - 2026
DA - 2026/10/30
PB - medRxiv
DO - 10.64898/2026.10.30.26366942
UR - https://www.medrxiv.org/content/10.64898/2026.10.30.26366942v1
AB - Interstitial lung disease (ILD) is a potentially fatal toxicity of immune checkpoint inhibitors and antibody-drug conjugates. Drug-safety studies in healthcare databases identify ILD with claims-based algorithms that trade positive predictive value (PPV) against sensitivity. We assessed whether large language models (LLMs) improve on these algorithms, with and without electronic health record (EHR) notes, and whether these gains translate into measurable epidemiological benefits. In 309 oncology patients with ILD diagnosis codes at Mass General Brigham, adjudicated by two pulmonologists, we compared 12 open-access LLMs with 20 published claims-based algorithms. LLMs received claims, EHR notes, or both, without task-specific training. Measures were PPV and yield (proportion classified positive). We projected the attenuation of a drug-induced ILD odds ratio under non-differential outcome misclassification. LLMs extended the PPV-yield frontier of the published algorithms with all three inputs. Across the 36 model--input evaluations, LLM PPV exceeded that of the best algorithm without specialist-coding requirements (Algorithm 4) by 14.3 percentage points on average. With claims alone, Qwen3-8B achieved 89.8% PPV (95% CI 78.2-95.6) at 15.9% yield, versus 86.7% at 4.9% for the highest-PPV algorithm. With notes alone, Qwen3-32B achieved 78.9% PPV at 49.2% yield, versus 64.6% at 51.1% for Algorithm~4. With combined claims and notes, Qwen3-14B achieved 100.0% PPV at 24.3% yield. Under a simulated drug-safety scenario with a true odds ratio of 3.0, the projected observed odds ratio was 2.67 with combined inputs versus 1.07 with Algorithm 4, corresponding to a 46-fold difference in the estimated sample size required for 80% power (591 versus 27,200 patients). Zero-shot LLMs offer a higher-PPV alternative to rule-based claims phenotyping for ILD, on all three input types. Because the claims-only configuration uses the same claims data as existing algorithms, it could be deployed in administrative databases without linked notes, and may recover drug-safety signal otherwise lost to outcome misclassification.
ER -