Can Large Language Models Diagnose Primary Immunodeficiency from Patient-Described Symptoms?
- 1Boston Children's Hospital
Abstract
Patients with primary immunodeficiency (PI) face prolonged diagnostic delays and may increasingly turn to large language models (LLMs) to interpret their symptoms during this period. We evaluated whether an LLM could recognize PI from symptom descriptions derived from interviews with 21 PI patients. In prior work, we showed that GPT-4o identified PI in 96% of cases when prompted with physician-written patient histories (Rider et al., 2025). Here, when prompted with symptom descriptions in patients own words, GPT-5.2 identified PI in only 7 cases (33%), although it suggested general immune system concerns in 17 cases (81%). This expert-versus-patient gap may indicate that LLMs are sensitive to the language and framing of symptom descriptions, performing substantially worse when patients describe their own symptoms in everyday language than when clinicians summarize patient histories in structured medical terms. This study underscores the need to carefully evaluate how LLMs are used in patient-facing applications.
Related
- Can Language Models Diagnose Sepsis? A Head-to-Head Evaluation Against 353 Physicians
- Case Report: Contezolid-containing regimens for disseminated Bacillus Calmette–Guérin disease in three children with primary immunodeficiency
- Pediatric Mycobacterium bovis Infection: Clinical Spectrum, BCG-Related Disease, Primary Immunodeficiency, and a Comprehensive Review of the Literature.
- Associations of Mediterranean Diet Adherence with Cardiometabolic Outcomes, Psychological Symptoms, and Health-Related Quality of Life Among Adults Attending Primary Care Centres in Navarra, Spain: A Cross-Sectional Study
- Extracting Symptoms of Psychotic Disorders from Clinical Notes using Natural Language Processing.
BibTeX
@misc{reteig2026large,
title = {{Can Large Language Models Diagnose Primary Immunodeficiency from Patient-Described Symptoms?}},
author = {L. C. Reteig and S. Woloshin and P. J. Maglione and J. R. Farmer and M.-S. Ong},
howpublished = {medRxiv},
year = {2026},
month = may,
doi = {10.64898/2026.05.26.26353818},
url = {https://www.medrxiv.org/content/10.64898/2026.05.26.26353818v2},
note = {CC BY},
}
RIS
TY - UNPB
TI - Can Large Language Models Diagnose Primary Immunodeficiency from Patient-Described Symptoms?
AU - L. C. Reteig
AU - S. Woloshin
AU - P. J. Maglione
AU - J. R. Farmer
AU - M.-S. Ong
PY - 2026
DA - 2026/05/26
PB - medRxiv
DO - 10.64898/2026.05.26.26353818
UR - https://www.medrxiv.org/content/10.64898/2026.05.26.26353818v2
AB - Patients with primary immunodeficiency (PI) face prolonged diagnostic delays and may increasingly turn to large language models (LLMs) to interpret their symptoms during this period. We evaluated whether an LLM could recognize PI from symptom descriptions derived from interviews with 21 PI patients. In prior work, we showed that GPT-4o identified PI in 96% of cases when prompted with physician-written patient histories (Rider et al., 2025). Here, when prompted with symptom descriptions in patients own words, GPT-5.2 identified PI in only 7 cases (33%), although it suggested general immune system concerns in 17 cases (81%). This expert-versus-patient gap may indicate that LLMs are sensitive to the language and framing of symptom descriptions, performing substantially worse when patients describe their own symptoms in everyday language than when clinicians summarize patient histories in structured medical terms. This study underscores the need to carefully evaluate how LLMs are used in patient-facing applications.
ER -