Comorbidities of dementia in electronic health records: validation of a natural language processing app and comparison with data linkage
- 1NIHR Biomedical Research Centre: Maudsley, KCL, South London and Maudsley NHS Mental Health Trust
Abstract
Awareness of comorbidities in people with dementia is vital context for dementia research and clinical management. Natural Language Processing (NLP) applied to free text in the electronic health record (EHR) may help ascertain comorbidity presence. Our aim was to assess the validity of the MedCAT Comorbidity NLP app in a cohort of people with dementia.
We used text from the EHR of South London and Maudsley NHS Mental Health Trust via Clinical Records Interactive Search (CRIS), linked to structured data from local primary care (Lambeth DataNet) and hospital admission data in a cohort of 3,543 people with dementia. We evaluated MedCAT Comorbidity, examining internal validity via manual review of text and external validity by cross-validation against linked primary care and hospital data; and comparison of the predictive value of comorbidity measures for mortality.
Internal precision was highest for atrial fibrillation (100%) and lowest for diabetes (78%). Cross-validation with linked data varied substantially across conditions, with precision from 96% to 59% and recall 77% to 4%. The median number of comorbidities identified was two using any single source, or three when combined. MedCAT Comorbidity demonstrated predictive ability for mortality (HR 1.11 per condition), although the model fit was better using comorbidity from hospital admission data.
In people with dementia, the MedCAT Comorbidity app showed high precision for most conditions during internal validation. The variability of precision and recall on external validity assessment partly reflects incomplete documentation. Results from the MedCAT comorbidity app can assist researchers in understanding comorbidity of cohorts, particularly when primary care or hospital data is not available. Our findings highlight the need to develop quality assurance protocols that include internal and external validity checks before deploying general data extraction apps for specific clinical subpopulations.
Author SummaryPeople with dementia commonly have other long-term health conditions, and structured electronic health record data may not capture the whole picture. Natural language processing (NLP) may offer additional information but there is a shortage of validation studies. This study explored performance of the MedCAT NLP comorbidity app on full-text records regarding 3,543 people with dementia that had external linkage to structured data.
There were three steps of validation: firstly, internal precision against the original clinical text; secondly, compared against primary care record and hospital admission diagnosis codes; thirdly, examining the ability of comorbidity ascertainment to predict mortality. The results indicated some potential strengths and limitations of the NLP approach, which may vary by comorbidity condition. With improvements, the MedCAT comorbidity app could expand its utility in dementia research and extend into clinical scenarios. Other NLP apps may benefit from a similar validation process in clinical subgroups before deployment.
Related
- Diagnostic reversion in dementia care: A real‐world analysis of mild cognitive impairment diagnoses after dementia in a large electronic medical record system
- PCSK9 inhibitor use and risk of dementia and neurodegenerative diseases in adults aged 50 Years and older: a global electronic health record study
- Deep Learning-Based Versus Threshold-Based Bone Removal for CT Angiography of Head and Neck Arteriovenous Fistulas: A Retrospective Comparison of Image Quality, Key-Structure Display, and Post-Processing Time
- Validation of an electronic-nose sensor array in detecting C. difficile and differentiating toxin-positive from toxin-negative stool samples
- Uncoded Clinical Features from Multilingual Electronic Health Records in Catalonia: Development and Validation Study
BibTeX
@misc{mirza2026comorbidities,
title = {{Comorbidities of dementia in electronic health records: validation of a natural language processing app and comparison with data linkage}},
author = {L. Mirza and R. Bendayan and R. Stewart and K. A. S. Davis},
howpublished = {medRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.16.26365714},
url = {https://www.medrxiv.org/content/10.64898/2026.10.16.26365714v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Comorbidities of dementia in electronic health records: validation of a natural language processing app and comparison with data linkage
AU - L. Mirza
AU - R. Bendayan
AU - R. Stewart
AU - K. A. S. Davis
PY - 2026
DA - 2026/10/16
PB - medRxiv
DO - 10.64898/2026.10.16.26365714
UR - https://www.medrxiv.org/content/10.64898/2026.10.16.26365714v1
AB - Awareness of comorbidities in people with dementia is vital context for dementia research and clinical management. Natural Language Processing (NLP) applied to free text in the electronic health record (EHR) may help ascertain comorbidity presence. Our aim was to assess the validity of the MedCAT Comorbidity NLP app in a cohort of people with dementia. We used text from the EHR of South London and Maudsley NHS Mental Health Trust via Clinical Records Interactive Search (CRIS), linked to structured data from local primary care (Lambeth DataNet) and hospital admission data in a cohort of 3,543 people with dementia. We evaluated MedCAT Comorbidity, examining internal validity via manual review of text and external validity by cross-validation against linked primary care and hospital data; and comparison of the predictive value of comorbidity measures for mortality. Internal precision was highest for atrial fibrillation (100%) and lowest for diabetes (78%). Cross-validation with linked data varied substantially across conditions, with precision from 96% to 59% and recall 77% to 4%. The median number of comorbidities identified was two using any single source, or three when combined. MedCAT Comorbidity demonstrated predictive ability for mortality (HR 1.11 per condition), although the model fit was better using comorbidity from hospital admission data. In people with dementia, the MedCAT Comorbidity app showed high precision for most conditions during internal validation. The variability of precision and recall on external validity assessment partly reflects incomplete documentation. Results from the MedCAT comorbidity app can assist researchers in understanding comorbidity of cohorts, particularly when primary care or hospital data is not available. Our findings highlight the need to develop quality assurance protocols that include internal and external validity checks before deploying general data extraction apps for specific clinical subpopulations. Author SummaryPeople with dementia commonly have other long-term health conditions, and structured electronic health record data may not capture the whole picture. Natural language processing (NLP) may offer additional information but there is a shortage of validation studies. This study explored performance of the MedCAT NLP comorbidity app on full-text records regarding 3,543 people with dementia that had external linkage to structured data. There were three steps of validation: firstly, internal precision against the original clinical text; secondly, compared against primary care record and hospital admission diagnosis codes; thirdly, examining the ability of comorbidity ascertainment to predict mortality. The results indicated some potential strengths and limitations of the NLP approach, which may vary by comorbidity condition. With improvements, the MedCAT comorbidity app could expand its utility in dementia research and extend into clinical scenarios. Other NLP apps may benefit from a similar validation process in clinical subgroups before deployment.
ER -