Skip to content
DraconiticResearch
Search Sources API
181,686 studies31 sourcesUpdated 4 min ago
Peer-reviewedCC BY 4.0Frontiers in Endocrinology

Application of LLM combined with knowledge base in improving C-TIRADS grading of thyroid nodules

  • Qian Qu1
  • Hanwen Bi2
  • Xiaofei Yu1
  • Rui Zhou3
  • Zhen Liu4
  • Tengteng Hao3
4 affiliations
  1. 1School of Traditional Chinese Medicine, Shandong Medical And Pharmaceutical University
  2. 2Department of Ultrasound, Yantai Fushan District People's Hospital
  3. 3Department of Oncology, Xiyuan Hospital of China Academy of Chinese Medical Sciences
  4. 4Department of Ultrasound Medicine, The Affiliated Yantai Yuhuangding Hospital of Qingdao University
Read full textPDFDOICite

Abstract

Objective: This study used the unified text-based grading results of senior sonographers as the primary reference (this text benchmark is not pathological diagnostic gold standard) to evaluate the concordance of text-derived C-TIRADS grades assigned by six large language models (LLMs), with and without FastGPT Retrieval-Augmented Generation (RAG) knowledge base assistance. Our primary endpoint was the concordance between grading and physician consensus—a measure fundamentally distinct from pathological diagnostic accuracy. Only a small subgroup with pathological confirmation was used for a supplementary exploratory analysis of the correlation between model risk stratification and pathological benign or malignant findings.

Methods: Six LLMs were configured using the C-TIRADS criteria. A FastGPT knowledge base was built from 11,489 thyroid ultrasound records collected at Yantai Yuhuangding Hospital between 2022 and 2023. Cases were assigned to manual, LLM, and knowledge-base-augmented LLM groups. C-TIRADS grading concordance with expert ultrasound consensus, and grading speed were assessed. A subgroup of 62 pathologically confirmed nodules was used for supplementary exploratory analysis. AUC, sensitivity, specificity, PPV, NPV and Mantel-Haenszel linear trend χ2 were computed to examine the alignment of model C-TIRADS risk tiers with pathological benign and malignant findings.

Results: Among the standalone LLMs, DeepSeek-V3 achieved the highest numerical grading concordance (81.14%). Following the integration of the RAG knowledge base, overall grading consistency improved significantly ( P < 0.001), with Doubao-pro-128k demonstrating the highest overall concordance at 89.04%. Concordance increased significantly for C-TIRADS Categories 1 and 3 ( P = 0.027 and P < 0.001, respectively), increased numerically for Category 2 ( P = 0.596), and decreased numerically for borderline Category 4A ( P = 0.106). The average concordance across all categories rose significantly from 70.39% to 79.75% ( P < 0.001). The knowledge-augmented Doubao-pro-128k model exhibited excellent inter-rater agreement ( linear-weighted Kappa>0.8, Bonferroni-corrected P<0.006 ). The average single-case grading duration of all 12 AI risk tiering pipelines (6 LLMs with/without knowledge base) was 1.30 s, which was 15 times faster than manual grading (19.25 s). Stratified trend and ROC analyses were performed on 62 thyroid nodules with pathological outcomes. Within this small-sample exploratory pathological subgroup, the Doubao-pro-128k integrated with knowledge base established a significantly ascending malignant risk gradient, achieving an AUC of 0.783 (95% CI : 0.635, 0.932) for distinguishing benign from malignant lesions. In comparison, knowledge-enhanced Doubao-pro-128k exhibited a trend toward better matching trend with pathological malignancy distribution and numerically the highest positive Youden index of 0.277 (95% CI : 0.071, 0.482).

Conclusion: Doubao-pro-128k integrated with knowledge base accelerated text grading and improved overall sonographer concordance; category-specific concordance increased significantly for C-TIRADS 1 and 3, increased numerically for Category 2, and declined numerically without statistical significance for borderline C-TIRADS 4A, indicating that these ambiguous nodules require further optimization. Exploratory pathological analysis revealed variable model-pathology alignment at the C-TIRADS ≥ 4A cutoff: DeepSeek-V3 integrated with knowledge base produced a negative Youden index, while Doubao-pro-128k integrated with knowledge base showed a tentative trend toward better reflected pathological malignancy gradients.

Related

  1. Determining malignancy in thyroid nodules featuring anterior macrocalcifications with strong posterior acoustic shadowing: the combined utility of US-FNA and BRAF V600E mutation analysis
    Jinjin Yang, Yongjie Xu, Jinshu Zeng +3
    Quantitative imaging in medicine and surgeryPeer-reviewedCC BY-NC-NDFull textMedical imaging & radiologyEurope PMC
  2. Salivary gland dynamic imaging combined with tomosynthesis fusion imaging for evaluating postoperative salivary gland function changes in patients with differentiated thyroid cancer undergoing initial 131I therapy
    Yanni Yang, Aijuan Tian, Jing Yu +2
    Quantitative imaging in medicine and surgeryPeer-reviewedCC BY-NC-NDFull textMedical imaging & radiologyEurope PMC
  3. A combined genetic and bioinformatic analysis implicates the CDKN1A rs1059234 variant in differentiated thyroid carcinoma susceptibility
    Elisangela de Souza Teixeira, Raquel Bueno Barbieri, Larissa Teodoro Rabi +4
    Archives of endocrinology and metabolismPeer-reviewedCC BYFull textEndocrinology & metabolismEurope PMC
  4. The Effect of 3D Printer and Moulage Use on Improving Stoma Care Knowledge and Skills in Nursing Students: A Randomized Controlled Trial
    Cagla Avcu, Tülin Yildiz
    International wound journalPeer-reviewedCC BY-NC-NDFull textGeneral medicineEurope PMC
  5. Effect of the end-of-life care education program on improving oncology nurses’ knowledge, attitudes, and behaviors: A quasi-experimental study
    Wenhua Yu, Xiaoting Hou, Renxiu Guo +4
    iSciencePeer-reviewedCC BY-NC-ND 4.0Full textEpidemiology & public healthCell PressCrossrefEurope PMC

Cite

BibTeXRIS
BibTeX
@article{qu2026application,
  title = {{Application of LLM combined with knowledge base in improving C-TIRADS grading of thyroid nodules}},
  author = {Qian Qu and Hanwen Bi and Xiaofei Yu and Rui Zhou and Zhen Liu and Tengteng Hao},
  journal = {Frontiers in Endocrinology},
  year = {2026},
  month = oct,
  doi = {10.3389/fendo.2026.1888772},
  url = {https://www.frontiersin.org/articles/10.3389/fendo.2026.1888772/full},
  note = {CC BY 4.0},
}
RIS
TY  - JOUR
TI  - Application of LLM combined with knowledge base in improving C-TIRADS grading of thyroid nodules
AU  - Qian Qu
AU  - Hanwen Bi
AU  - Xiaofei Yu
AU  - Rui Zhou
AU  - Zhen Liu
AU  - Tengteng Hao
PY  - 2026
DA  - 2026/10/15
JO  - Frontiers in Endocrinology
DO  - 10.3389/fendo.2026.1888772
UR  - https://www.frontiersin.org/articles/10.3389/fendo.2026.1888772/full
AB  - Objective: This study used the unified text-based grading results of senior sonographers as the primary reference (this text benchmark is not pathological diagnostic gold standard) to evaluate the concordance of text-derived C-TIRADS grades assigned by six large language models (LLMs), with and without FastGPT Retrieval-Augmented Generation (RAG) knowledge base assistance. Our primary endpoint was the concordance between grading and physician consensus—a measure fundamentally distinct from pathological diagnostic accuracy. Only a small subgroup with pathological confirmation was used for a supplementary exploratory analysis of the correlation between model risk stratification and pathological benign or malignant findings.  Methods: Six LLMs were configured using the C-TIRADS criteria. A FastGPT knowledge base was built from 11,489 thyroid ultrasound records collected at Yantai Yuhuangding Hospital between 2022 and 2023. Cases were assigned to manual, LLM, and knowledge-base-augmented LLM groups. C-TIRADS grading concordance with expert ultrasound consensus, and grading speed were assessed. A subgroup of 62 pathologically confirmed nodules was used for supplementary exploratory analysis. AUC, sensitivity, specificity, PPV, NPV and Mantel-Haenszel linear trend χ2 were computed to examine the alignment of model C-TIRADS risk tiers with pathological benign and malignant findings.  Results: Among the standalone LLMs, DeepSeek-V3 achieved the highest numerical grading concordance (81.14%). Following the integration of the RAG knowledge base, overall grading consistency improved significantly ( P < 0.001), with Doubao-pro-128k demonstrating the highest overall concordance at 89.04%. Concordance increased significantly for C-TIRADS Categories 1 and 3 ( P = 0.027 and P < 0.001, respectively), increased numerically for Category 2 ( P = 0.596), and decreased numerically for borderline Category 4A ( P = 0.106). The average concordance across all categories rose significantly from 70.39% to 79.75% ( P < 0.001). The knowledge-augmented Doubao-pro-128k model exhibited excellent inter-rater agreement ( linear-weighted Kappa>0.8, Bonferroni-corrected P<0.006 ). The average single-case grading duration of all 12 AI risk tiering pipelines (6 LLMs with/without knowledge base) was 1.30 s, which was 15 times faster than manual grading (19.25 s). Stratified trend and ROC analyses were performed on 62 thyroid nodules with pathological outcomes. Within this small-sample exploratory pathological subgroup, the Doubao-pro-128k integrated with knowledge base established a significantly ascending malignant risk gradient, achieving an AUC of 0.783 (95% CI : 0.635, 0.932) for distinguishing benign from malignant lesions. In comparison, knowledge-enhanced Doubao-pro-128k exhibited a trend toward better matching trend with pathological malignancy distribution and numerically the highest positive Youden index of 0.277 (95% CI : 0.071, 0.482).  Conclusion: Doubao-pro-128k integrated with knowledge base accelerated text grading and improved overall sonographer concordance; category-specific concordance increased significantly for C-TIRADS 1 and 3, increased numerically for Category 2, and declined numerically without statistical significance for borderline C-TIRADS 4A, indicating that these ambiguous nodules require further optimization. Exploratory pathological analysis revealed variable model-pathology alignment at the C-TIRADS ≥ 4A cutoff: DeepSeek-V3 integrated with knowledge base produced a negative Youden index, while Doubao-pro-128k integrated with knowledge base showed a tentative trend toward better reflected pathological malignancy gradients.
SN  - 1664-2392
ER  -