Diagnostic Value of Large Language Model-Extracted Gross Brain Findings in Neurodegenerative Diseases
- 1Hospital of the University of Pennsylvania
Abstract
Gross brain findings help guide differential diagnosis, but their diagnostic value across major neurodegenerative diseases remains incompletely characterized. We evaluated whether gross descriptions in autopsy reports could classify cases by final neuropathologic diagnosis. We analyzed 5,613 autopsy cases from the Mayo Clinic Brain Bank collected between 1998 and 2023, including seven diagnostic categories: Alzheimer disease (AD), Lewy body disease (LBD), combined AD and LBD (AD-LBD), frontotemporal lobar degeneration, corticobasal degeneration, progressive supranuclear palsy (PSP), and multiple system atrophy (MSA). A fine-tuned large language model (LLM) converted narrative gross descriptions into semi-quantitative scores for 39 features. Extraction accuracy was 0.95 in 200 manually annotated feature-level test examples. A CatBoost classifier used the extracted scores as input, whereas a second fine-tuned LLM used standardized textual descriptions of the same features. Both classifiers also included age at death, sex, and brain weight. Performance was evaluated in the same held-out test set of 562 cases. CatBoost achieved an accuracy of 0.73, a Cohens kappa of 0.65, and a macro-average area under the receiver operating characteristic curve of 0.92. The text-based LLM achieved an accuracy of 0.75 and a kappa of 0.68. Macro-average sensitivity was 0.66 for both models. PSP sensitivity was 0.92 with CatBoost and 0.93 with text-based LLM. Corresponding sensitivities were 0.87 and 0.92 for MSA, but only 0.21 and 0.03 for AD-LBD. Feature attribution analysis identified subthalamic nucleus atrophy and putaminal abnormalities as contributors to PSP and MSA predictions, respectively. Narrative gross descriptions contained diagnostic information, particularly for disorders with distinctive macroscopic patterns, whereas combined AD-LBD remained difficult to distinguish.
Related
- Diagnostic Accuracy of Fine Needle Aspiration Cytology in Thyroid Diseases: Comparison with Histopathological Findings in Hawassa, Ethiopia
- Modeling Strategies for Neurodegenerative Diseases: In Vitro, In Vivo and In Silico Frameworks
- Editorial: Glucose metabolism in brain and neurodegenerative diseases: multifactorial effects
- Brain-computer interfaces for neurodegenerative diseases: A comprehensive review of closed-loop systems from diagnosis to neural rehabilitation
- Diagnosing Neurodegenerative Diseases by Label-Free 3-D Imaging of Intestinal Samples
BibTeX
@misc{ono2026diagnostic,
title = {{Diagnostic Value of Large Language Model-Extracted Gross Brain Findings in Neurodegenerative Diseases}},
author = {D. Ono and D. W. Dickson and S. Koga},
howpublished = {medRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.12.26362866},
url = {https://www.medrxiv.org/content/10.64898/2026.09.12.26362866v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Diagnostic Value of Large Language Model-Extracted Gross Brain Findings in Neurodegenerative Diseases
AU - D. Ono
AU - D. W. Dickson
AU - S. Koga
PY - 2026
DA - 2026/09/12
PB - medRxiv
DO - 10.64898/2026.09.12.26362866
UR - https://www.medrxiv.org/content/10.64898/2026.09.12.26362866v1
AB - Gross brain findings help guide differential diagnosis, but their diagnostic value across major neurodegenerative diseases remains incompletely characterized. We evaluated whether gross descriptions in autopsy reports could classify cases by final neuropathologic diagnosis. We analyzed 5,613 autopsy cases from the Mayo Clinic Brain Bank collected between 1998 and 2023, including seven diagnostic categories: Alzheimer disease (AD), Lewy body disease (LBD), combined AD and LBD (AD-LBD), frontotemporal lobar degeneration, corticobasal degeneration, progressive supranuclear palsy (PSP), and multiple system atrophy (MSA). A fine-tuned large language model (LLM) converted narrative gross descriptions into semi-quantitative scores for 39 features. Extraction accuracy was 0.95 in 200 manually annotated feature-level test examples. A CatBoost classifier used the extracted scores as input, whereas a second fine-tuned LLM used standardized textual descriptions of the same features. Both classifiers also included age at death, sex, and brain weight. Performance was evaluated in the same held-out test set of 562 cases. CatBoost achieved an accuracy of 0.73, a Cohens kappa of 0.65, and a macro-average area under the receiver operating characteristic curve of 0.92. The text-based LLM achieved an accuracy of 0.75 and a kappa of 0.68. Macro-average sensitivity was 0.66 for both models. PSP sensitivity was 0.92 with CatBoost and 0.93 with text-based LLM. Corresponding sensitivities were 0.87 and 0.92 for MSA, but only 0.21 and 0.03 for AD-LBD. Feature attribution analysis identified subthalamic nucleus atrophy and putaminal abnormalities as contributors to PSP and MSA predictions, respectively. Narrative gross descriptions contained diagnostic information, particularly for disorders with distinctive macroscopic patterns, whereas combined AD-LBD remained difficult to distinguish.
ER -