Natural Language Processing Based Solution for Labeling Brain Metastasis Identified in Radiology Reports
- 1University of Alberta
Abstract
PurposeBrain metastases (BM) far exceed primary CNS tumours and constitute the majority workload for neuro-oncology care providers. Currently, the cancer registries only capture synchronous BMs, which is only a small proportion of all BMs. We aim to develop and validate a natural language processing (NLP) algorithm that identifies brain metastases in radiology reports, enabling scalable surveillance of asynchronous BMs.
MethodsUsing population-based cancer registry data in Alberta, Canada, we identified a cancer cohort diagnosed between 2012-2019 with follow-up to 2022. All brain/head radiology reports at and post-cancer diagnosis were identified. Reports were sampled through a multi-phase approach and manually labeled for BM presence. We trained two Bio_ClinicalBERT models on the "Findings" and "Impressions" sections, respectively, and took the maximum predicted probability as the report-level prediction. Internal and external validation used reports from the Canadian provinces of Alberta, Ontario, and British Columbia.
ResultsThe models were trained on 1,879 samples. For internal validation, 1,833 reports from 357 patients were tested. At a probability threshold of 0.4, the model achieved a sensitivity of 0.888 and precision of 0.499. The ensemble substantially outperformed single-section models, which achieved sensitivities of only 67.8% (Findings) and 74.2% (Impressions). On external validation, sensitivity was 0.918 in Ontario and 0.726 in British Columbia, demonstrating robustness across diverse data distributions.
ConclusionsAn NLP-based pipeline processing both Findings and Impressions sections has been developed and validated in three Canadian provinces. It meets cancer registry operational requirements and to be implemented into the surveillance workflow in Alberta and British Columbia, providing a foundation for population-level BM surveillance.
Related
- Sycophancy in Large Language Models in Oral and Maxillofacial Radiology: The Impact of Misleading Context on Image- and Text-Based Diagnostic Scenarios
- Knowledge distillation-based lightweight generative large language models for standardized quality control of Chinese radiology reports
- Large Language Models in the Radiology Workflow: Current Applications, Challenges, and Future Perspectives.
- Enhancing medical education with AI: A comparative study of various large language models in radiology
- Large language model consensus for reliable research cohort construction from radiology reports: a retrospective cohort study
BibTeX
@misc{liu2026natural,
title = {{Natural Language Processing Based Solution for Labeling Brain Metastasis Identified in Radiology Reports}},
author = {T. Liu and Y. T. Han and H. Zuo and S. Das and H.-M. Lin and E. Colak and M. Istasy and A. M. Ladak and J. C. Bigenimana and L. Gondara and J. Simkin and J. Lee and D. Roozbeh and A. M. Nichol and J. Easaw and E. Walker and S. Yip and L. Mou and Y. Yuan},
howpublished = {medRxiv},
year = {2026},
month = jun,
doi = {10.64898/2026.06.10.26355415},
url = {https://www.medrxiv.org/content/10.64898/2026.06.10.26355415v2},
note = {CC BY},
}
RIS
TY - UNPB
TI - Natural Language Processing Based Solution for Labeling Brain Metastasis Identified in Radiology Reports
AU - T. Liu
AU - Y. T. Han
AU - H. Zuo
AU - S. Das
AU - H.-M. Lin
AU - E. Colak
AU - M. Istasy
AU - A. M. Ladak
AU - J. C. Bigenimana
AU - L. Gondara
AU - J. Simkin
AU - J. Lee
AU - D. Roozbeh
AU - A. M. Nichol
AU - J. Easaw
AU - E. Walker
AU - S. Yip
AU - L. Mou
AU - Y. Yuan
PY - 2026
DA - 2026/06/10
PB - medRxiv
DO - 10.64898/2026.06.10.26355415
UR - https://www.medrxiv.org/content/10.64898/2026.06.10.26355415v2
AB - PurposeBrain metastases (BM) far exceed primary CNS tumours and constitute the majority workload for neuro-oncology care providers. Currently, the cancer registries only capture synchronous BMs, which is only a small proportion of all BMs. We aim to develop and validate a natural language processing (NLP) algorithm that identifies brain metastases in radiology reports, enabling scalable surveillance of asynchronous BMs. MethodsUsing population-based cancer registry data in Alberta, Canada, we identified a cancer cohort diagnosed between 2012-2019 with follow-up to 2022. All brain/head radiology reports at and post-cancer diagnosis were identified. Reports were sampled through a multi-phase approach and manually labeled for BM presence. We trained two Bio_ClinicalBERT models on the "Findings" and "Impressions" sections, respectively, and took the maximum predicted probability as the report-level prediction. Internal and external validation used reports from the Canadian provinces of Alberta, Ontario, and British Columbia. ResultsThe models were trained on 1,879 samples. For internal validation, 1,833 reports from 357 patients were tested. At a probability threshold of 0.4, the model achieved a sensitivity of 0.888 and precision of 0.499. The ensemble substantially outperformed single-section models, which achieved sensitivities of only 67.8% (Findings) and 74.2% (Impressions). On external validation, sensitivity was 0.918 in Ontario and 0.726 in British Columbia, demonstrating robustness across diverse data distributions. ConclusionsAn NLP-based pipeline processing both Findings and Impressions sections has been developed and validated in three Canadian provinces. It meets cancer registry operational requirements and to be implemented into the surveillance workflow in Alberta and British Columbia, providing a foundation for population-level BM surveillance.
ER -