Impression-regularized finetuning enables small language models to extract cervical spine fractures from radiology reports
7 affiliations
- 1Department of Electrical and Computer Engineering, University of Toronto, Toronto, Ontario, Canada.
- 2Department of Medical Imaging, St Michael's Hospital, Unity Health Toronto, Toronto, Ontario, Canada.
- 3Department of Medical Imaging, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada.
- 4Division of Neurosurgery, Department of Surgery, University of Toronto, Toronto, Ontario, Canada.
- 5Department of Computer Science, Emory University, Atlanta, Georgia, United States of America.
- 6Research and Innovation, North York General Hospital, Toronto, Ontario, Canada.
- 7Li Ka Shing Knowledge Institute, St Michael's Hospital, Unity Health Toronto, Toronto, Ontario, Canada.
Abstract
We assessed whether small, open-weight large language models (LLMs) can accurately extract vertebral-level acute cervical spine fracture labels from CT radiology reports while supporting on-premises deployment, and whether impression-regularized finetuning improves accuracy, data efficiency, and temporal stability. In this single-center retrospective study, 8,567 adult cervical spine CT reports (2017–2022) were labeled for acute occipital condyle and C1–C7 fractures with neuroradiologist oversight. Reports from 2017 (n = 1,064) were used for finetuning and reports from 2018–2022 (n = 7,503) for temporal testing. We finetuned Gemma-3 (1B/4B/12B), Llama-3.2-3B, and Phi-3.5-Mini using standard low-rank adaptation (LoRA) and with an impression-regularized LoRA approach. During training, the model samples multiple synthetic targeted impressions focused on acute cervical spine fractures, selects the candidate most consistent with the reference standard labels, and applies an auxiliary impression loss as regularization. At inference, all models perform single-pass deterministic JSON extraction. Baselines included zero-shot prompting, two-pass impression-guided prompting, and standard LoRA. Impression-regularized LoRA improved Gemma-3-4B macro-F1 from 0.885 to 0.909 (macro-precision 0.929, macro-recall 0.893) while reducing variance across training runs. Improvements were largest for smaller models and in low-data settings; Gemma-3-1B macro-F1 increased from 0.646 to 0.782. For this vertebral-level acute cervical spine fracture extraction task, impression-regularized Gemma-3-4B approached the zero-shot performance of the Gemma-3-27B (macro-F1 0.909 vs. 0.913). Across the 2018–2022 temporal evaluation, mean F1 increased from 0.871 to 0.893 with similar or lower variability across time windows. Systematic error analysis showed that remaining errors were mainly associated with ambiguous report language and label-boundary cases. Overall, impression-regularized LoRA enables small open-weight LLMs to accurately extract acute cervical spine fracture labels while maintaining performance across a multi-year test period and preserving auditable single-pass inference suitable for privacy-preserving on-premises deployment. Author summary: Radiology reports contain important details about injuries, but these details are often written in free text rather than in a format that can be easily searched, counted, or used for quality improvement. In this study, we examined whether smaller language models can extract structured information about acute cervical spine fractures from computed tomography reports, including which specific anatomic level is involved. Our approach trains the model not only to produce the final structured labels, but also to learn from a short, report-like summary of the relevant findings during training. This added supervision is not intended to be a clinical explanation or a replacement for radiologist review. Instead, it provides an additional learning signal that may help the model focus on fracture location, acuity, and negation. We found that this approach improved performance in our single-center dataset, particularly for smaller models and limited training data. Performance was also maintained across reports from later years in the same institution. These findings suggest that locally deployable language models may help support retrospective cohort identification, monitoring, or non-blocking workflow tools. However, external validation, prospective monitoring, and careful attention to errors and false alerts are needed before broader clinical use.
Related
- Enhancing medical education with AI: A comparative study of various large language models in radiology
- Targeted finetuning enables co-folding models to learn ligand-induced protein conformational states
- The neurocomputational mechanisms of naturalistic trait impression updating
- Interspecies Differential Gene Expression Analysis with Regularized Phylogenetic Linear Models
- Regularized logistic regression-based gene selection for functional analysis of heterogeneous scRNA-seq macrophage populations in muscle aging and regeneration.
BibTeX
@article{hu2026impressionregularized,
title = {{Impression-regularized finetuning enables small language models to extract cervical spine fractures from radiology reports}},
author = {Zixuan Hu and Hui Ming Lin and Yingming Amy Chen and Christopher S Lozano and Markand Patel and Ali Emami and Ervin Sejdić and Errol Colak},
journal = {PLOS Digital Health},
year = {2026},
month = oct,
doi = {10.1371/journal.pdig.0001786},
url = {https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001786},
note = {CC BY},
}
RIS
TY - JOUR
TI - Impression-regularized finetuning enables small language models to extract cervical spine fractures from radiology reports
AU - Zixuan Hu
AU - Hui Ming Lin
AU - Yingming Amy Chen
AU - Christopher S Lozano
AU - Markand Patel
AU - Ali Emami
AU - Ervin Sejdić
AU - Errol Colak
PY - 2026
DA - 2026/10/21
JO - PLOS Digital Health
DO - 10.1371/journal.pdig.0001786
UR - https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001786
AB - We assessed whether small, open-weight large language models (LLMs) can accurately extract vertebral-level acute cervical spine fracture labels from CT radiology reports while supporting on-premises deployment, and whether impression-regularized finetuning improves accuracy, data efficiency, and temporal stability. In this single-center retrospective study, 8,567 adult cervical spine CT reports (2017–2022) were labeled for acute occipital condyle and C1–C7 fractures with neuroradiologist oversight. Reports from 2017 (n = 1,064) were used for finetuning and reports from 2018–2022 (n = 7,503) for temporal testing. We finetuned Gemma-3 (1B/4B/12B), Llama-3.2-3B, and Phi-3.5-Mini using standard low-rank adaptation (LoRA) and with an impression-regularized LoRA approach. During training, the model samples multiple synthetic targeted impressions focused on acute cervical spine fractures, selects the candidate most consistent with the reference standard labels, and applies an auxiliary impression loss as regularization. At inference, all models perform single-pass deterministic JSON extraction. Baselines included zero-shot prompting, two-pass impression-guided prompting, and standard LoRA. Impression-regularized LoRA improved Gemma-3-4B macro-F1 from 0.885 to 0.909 (macro-precision 0.929, macro-recall 0.893) while reducing variance across training runs. Improvements were largest for smaller models and in low-data settings; Gemma-3-1B macro-F1 increased from 0.646 to 0.782. For this vertebral-level acute cervical spine fracture extraction task, impression-regularized Gemma-3-4B approached the zero-shot performance of the Gemma-3-27B (macro-F1 0.909 vs. 0.913). Across the 2018–2022 temporal evaluation, mean F1 increased from 0.871 to 0.893 with similar or lower variability across time windows. Systematic error analysis showed that remaining errors were mainly associated with ambiguous report language and label-boundary cases. Overall, impression-regularized LoRA enables small open-weight LLMs to accurately extract acute cervical spine fracture labels while maintaining performance across a multi-year test period and preserving auditable single-pass inference suitable for privacy-preserving on-premises deployment. Author summary: Radiology reports contain important details about injuries, but these details are often written in free text rather than in a format that can be easily searched, counted, or used for quality improvement. In this study, we examined whether smaller language models can extract structured information about acute cervical spine fractures from computed tomography reports, including which specific anatomic level is involved. Our approach trains the model not only to produce the final structured labels, but also to learn from a short, report-like summary of the relevant findings during training. This added supervision is not intended to be a clinical explanation or a replacement for radiologist review. Instead, it provides an additional learning signal that may help the model focus on fracture location, acuity, and negation. We found that this approach improved performance in our single-center dataset, particularly for smaller models and limited training data. Performance was also maintained across reports from later years in the same institution. These findings suggest that locally deployable language models may help support retrospective cohort identification, monitoring, or non-blocking workflow tools. However, external validation, prospective monitoring, and careful attention to errors and false alerts are needed before broader clinical use.
SN - 2767-3170
ER -