A cross-sectional benchmark of seven large language models for National Comprehensive Cancer Network guideline adherence in non-small cell lung cancer management.

@article{sun2026crosssectional,
  title = {{A cross-sectional benchmark of seven large language models for National Comprehensive Cancer Network guideline adherence in non-small cell lung cancer management.}},
  author = {Zewen Sun and Tian Guan and Jiayi Geng and Junwei Liu and Dexin Su and Tao Sun and Gangzeng Jin and Heng Zhao and Shaodong Wang and Geyun Chang and Guihu Lin and Changtai Zhao and Kexin Feng and Hongchengcheng Chen and Jiaming Deng and Ruzhen Li and Jun Wang and Peihao Li and Liqi Liu and Yun Li and Fan Yang and Jun Wang and Xin Wang and Ailing Yu and Yuan Xia and Dongjie Tao and Hualei Zhou and Chunxiao Guo and Peng Wei and Jian Wang and Jinjie Gu and Yanyun Wang and Jing Zhao and Shuai Zhen and Kezhong Chen and Ping Wang and Xiuyuan Chen},
  journal = {Translational lung cancer research},
  year = {2026},
  month = sep,
  doi = {10.21037/tlcr-2026-0698},
  url = {https://europepmc.org/article/MED/42999632},
  note = {CC BY-NC-ND},
}

TY  - JOUR
TI  - A cross-sectional benchmark of seven large language models for National Comprehensive Cancer Network guideline adherence in non-small cell lung cancer management.
AU  - Zewen Sun
AU  - Tian Guan
AU  - Jiayi Geng
AU  - Junwei Liu
AU  - Dexin Su
AU  - Tao Sun
AU  - Gangzeng Jin
AU  - Heng Zhao
AU  - Shaodong Wang
AU  - Geyun Chang
AU  - Guihu Lin
AU  - Changtai Zhao
AU  - Kexin Feng
AU  - Hongchengcheng Chen
AU  - Jiaming Deng
AU  - Ruzhen Li
AU  - Jun Wang
AU  - Peihao Li
AU  - Liqi Liu
AU  - Yun Li
AU  - Fan Yang
AU  - Jun Wang
AU  - Xin Wang
AU  - Ailing Yu
AU  - Yuan Xia
AU  - Dongjie Tao
AU  - Hualei Zhou
AU  - Chunxiao Guo
AU  - Peng Wei
AU  - Jian Wang
AU  - Jinjie Gu
AU  - Yanyun Wang
AU  - Jing Zhao
AU  - Shuai Zhen
AU  - Kezhong Chen
AU  - Ping Wang
AU  - Xiuyuan Chen
PY  - 2026
DA  - 2026/09/29
JO  - Translational lung cancer research
DO  - 10.21037/tlcr-2026-0698
UR  - https://europepmc.org/article/MED/42999632
AB  - While large language models (LLMs) show promise in general medicine, existing benchmarks lack the disease-specific rigor needed to assess guideline adherence. Non-small cell lung cancer (NSCLC) care requires alignment with rapidly evolving National Comprehensive Cancer Network (NCCN) Guidelines. We developed an NCCN-grounded benchmark to assess guideline concordance of seven selected LLMs in NSCLC management. We developed an NCCN (Version 4.2025)-grounded NSCLC benchmark. The final analysis set contained 793 archived question-and-answer (Q&A) items. Each item included weighted positive criteria and safety-critical negative criteria. Seven specified LLMs were evaluated once per item, without a system prompt, retrieval augmentation, tools, or internet access. Automated rubric-based scores were used for descriptive comparison. Across seven specified LLMs on 793 NCCN Guidelines-based queries, Gemini-3-Pro achieved the highest clean normalized score (0.739), while DeepSeek-V3.2 was the highest-scoring open-source model (0.695). Scores were generally higher for foundational tasks and lower for clinical-trial interpretation and therapeutic modality selection, with interpretation of clinical trial results showing the lowest task-level performance. Gemini-3-Pro and DeepSeek-V3.2 had the lowest omission rates (28.36% and 33.24%, respectively). Gemini-3-Pro had a 0% inclusion rate of rubric-defined contraindicated negative points in this benchmark. Automated scores are benchmark-specific and do not establish clinical safety or independently validated clinical quality. This NCCN-grounded benchmark compares seven specified LLMs using a fixed question set. Although leading models performed better on several foundational tasks, the results reflect guideline-rubric concordance rather than clinical correctness and do not show readiness for independent clinical decision-making.
SN  - 2218-6751
SN  - 2226-4477
ER  -