Skip to content
DraconiticResearch
Search Sources API
194,161 studies31 sourcesUpdated 15 min ago
Peer-reviewedCC BYFrontiers in cardiovascular medicine

Large language model-assisted CAD-RADS classification: structured report extraction, accuracy, and clinical decision-making risk of ChatGPT-4o/5 and DeepSeek-R1

  • Haorong Rong1
  • Xiyi Huang2
  • Xinjie Chen1
  • Lizhu Ouyang3
  • Yuelu Li1
  • Xin Lin1
  • Jialing Pan1
  • Yunjing Li1
  • Fan Xie1
  • Fusheng Ouyang1
  • Qiugen Hu1
  • Baoliang Guo1
  1. 1Department of Radiology, The Eighth Affiliated Hospital, Southern Medical University , , ,
  2. 2Department of Clinical Laboratory, Lecong Hospital of Shunde , , ,
  3. 3Department of Ultrasound, The Eighth Affiliated Hospital, Southern Medical University , , ,
Read full textPDFDOICite

Abstract

Objective: To compare ChatGPT-4o, ChatGPT-5, and DeepSeek-R1 in assigning main Coronary Artery Disease Reporting and Data System (CAD-RADS) categories from findings-only coronary computed tomography angiography (CCTA) reports and to characterize management-relevant classification errors.

Methods: This retrospective diagnostic-accuracy study included 2,100 category-balanced findings-only CCTA reports (300 per CAD-RADS category). Two experienced CCTA radiologists graded the English-translated reports; 12 disagreements were resolved by a third. ChatGPT-4o was evaluated in standard mode, ChatGPT-5 in Thinking mode, and DeepSeek-R1 in Deep Thinking mode; model generation and inference mode therefore varied together. Exact-match accuracy with 95% Wilson confidence intervals (CIs), paired comparisons, and unweighted Gwet AC1 with 95% CIs were calculated. Management-relevant transitions and digital subtraction angiography (DSA) findings among high-risk downgrades were analyzed.

Results: DeepSeek-R1 had an exact-match accuracy of 82.57% (95% CI, 80.89–84.13) and AC1 of 0.797 (95% CI, 0.778–0.816). ChatGPT-5 had an accuracy of 80.19% (95% CI, 78.43–81.84) and AC1 of 0.769 (95% CI, 0.749–0.789), whereas ChatGPT-4o had an accuracy of 53.62% (95% CI, 51.48–55.74) and AC1 of 0.460 (95% CI, 0.435–0.485). Cochran's Q test was significant (Q = 657.87; df = 2; P < 0.001); after Holm correction, DeepSeek-R1 also outperformed ChatGPT-5 in the paired comparison ( P = 0.026), and both outperformed ChatGPT-4o (both P < 0.001). ChatGPT-4o downgraded 127/900 (14.11%) reference high-risk reports to CAD-RADS 3 or lower. Of these 127 cases, 65 underwent DSA within three months; DSA supported the original high-risk assessment in 56/65 (86.15%). The corresponding high-risk downgrade proportions were 37/900 (4.11%) for ChatGPT-5 and 23/900 (2.56%) for DeepSeek-R1.

Conclusion: The DeepSeek-R1 Deep Thinking and ChatGPT-5 Thinking configurations achieved higher accuracy than ChatGPT-4o in standard mode. Because model generation and inference mode varied together, the results do not isolate a model-family effect. LLMs merit further evaluation for secondary structuring when no documented CAD-RADS category is available, but should not overwrite a radiologist-assigned category or independently determine management.

Related

  1. Zero-Shot Extraction of Structured Clinical Data from Free-Text CT Reports Using Foundational Large Language Models.
    Amulya Gupta, Alex Hongslo, Quynh Nguyen +7
    Kansas journal of medicinePeer-reviewedCC BY-NC-NDFull textGeneral medicineEurope PMC
  2. Non-generative readout with few-label calibration versus LoRA fine-tuning of a large language model for ASA physical status classification: accuracy, speed, cost and label efficiency in three data sources
    Y. Shi, Y. Wang, A. Simayi +2
    medRxivPreprintCC BY-NC-NDFull textSurgery & anaesthesiamedRxiv
  3. Fine-tuned Medical Vision-Language Model (MedGemma 1.5 4B) with and without Structured Radiologist Descriptors versus EU-TIRADS for Malignancy Risk Stratification of Thyroid Nodules at Ultrasound: A Tunisian Multicenter Diagnostic Accuracy Study
    M. N. BOU ALI, N. Lassoued, H. Mahmoud +5
    medRxivPreprintCC BY-NC-NDFull textMedical imaging & radiologymedRxiv
  4. LLM-Based Extraction of Clinical Practice Guidelines into Structured Arguments
    Robin Blouin, Karima Sedki, Jean-Baptiste Lamy
    Studies in Health Technology and InformaticsPeer-reviewedCC BY-NC 4.0Full textGeneral biologyEpidemiology & public healthHAL
  5. Robot-assisted total laparoscopic pancreaticoduodenectomy with transvaginal specimen extraction: a case report
    Journal of surgical case reportsPeer-reviewedCC BYFull textSurgery & anaesthesiaEurope PMC

Cite

BibTeXRIS
BibTeX
@article{rong2026large,
  title = {{Large language model-assisted CAD-RADS classification: structured report extraction, accuracy, and clinical decision-making risk of ChatGPT-4o/5 and DeepSeek-R1}},
  author = {Haorong Rong and Xiyi Huang and Xinjie Chen and Lizhu Ouyang and Yuelu Li and Xin Lin and Jialing Pan and Yunjing Li and Fan Xie and Fusheng Ouyang and Qiugen Hu and Baoliang Guo},
  journal = {Frontiers in cardiovascular medicine},
  year = {2026},
  month = oct,
  doi = {10.3389/fcvm.2026.1909068},
  url = {https://europepmc.org/article/PMC/PMC13690529},
  note = {CC BY},
}
RIS
TY  - JOUR
TI  - Large language model-assisted CAD-RADS classification: structured report extraction, accuracy, and clinical decision-making risk of ChatGPT-4o/5 and DeepSeek-R1
AU  - Haorong Rong
AU  - Xiyi Huang
AU  - Xinjie Chen
AU  - Lizhu Ouyang
AU  - Yuelu Li
AU  - Xin Lin
AU  - Jialing Pan
AU  - Yunjing Li
AU  - Fan Xie
AU  - Fusheng Ouyang
AU  - Qiugen Hu
AU  - Baoliang Guo
PY  - 2026
DA  - 2026/10/09
JO  - Frontiers in cardiovascular medicine
DO  - 10.3389/fcvm.2026.1909068
UR  - https://europepmc.org/article/PMC/PMC13690529
AB  - Objective: To compare ChatGPT-4o, ChatGPT-5, and DeepSeek-R1 in assigning main Coronary Artery Disease Reporting and Data System (CAD-RADS) categories from findings-only coronary computed tomography angiography (CCTA) reports and to characterize management-relevant classification errors.  Methods: This retrospective diagnostic-accuracy study included 2,100 category-balanced findings-only CCTA reports (300 per CAD-RADS category). Two experienced CCTA radiologists graded the English-translated reports; 12 disagreements were resolved by a third. ChatGPT-4o was evaluated in standard mode, ChatGPT-5 in Thinking mode, and DeepSeek-R1 in Deep Thinking mode; model generation and inference mode therefore varied together. Exact-match accuracy with 95% Wilson confidence intervals (CIs), paired comparisons, and unweighted Gwet AC1 with 95% CIs were calculated. Management-relevant transitions and digital subtraction angiography (DSA) findings among high-risk downgrades were analyzed.  Results: DeepSeek-R1 had an exact-match accuracy of 82.57% (95% CI, 80.89–84.13) and AC1 of 0.797 (95% CI, 0.778–0.816). ChatGPT-5 had an accuracy of 80.19% (95% CI, 78.43–81.84) and AC1 of 0.769 (95% CI, 0.749–0.789), whereas ChatGPT-4o had an accuracy of 53.62% (95% CI, 51.48–55.74) and AC1 of 0.460 (95% CI, 0.435–0.485). Cochran's Q test was significant (Q = 657.87; df = 2; P < 0.001); after Holm correction, DeepSeek-R1 also outperformed ChatGPT-5 in the paired comparison ( P = 0.026), and both outperformed ChatGPT-4o (both P < 0.001). ChatGPT-4o downgraded 127/900 (14.11%) reference high-risk reports to CAD-RADS 3 or lower. Of these 127 cases, 65 underwent DSA within three months; DSA supported the original high-risk assessment in 56/65 (86.15%). The corresponding high-risk downgrade proportions were 37/900 (4.11%) for ChatGPT-5 and 23/900 (2.56%) for DeepSeek-R1.  Conclusion: The DeepSeek-R1 Deep Thinking and ChatGPT-5 Thinking configurations achieved higher accuracy than ChatGPT-4o in standard mode. Because model generation and inference mode varied together, the results do not isolate a model-family effect. LLMs merit further evaluation for secondary structuring when no documented CAD-RADS category is available, but should not overwrite a radiologist-assigned category or independently determine management.
SN  - 2297-055X
ER  -