Skip to content
DraconiticResearch
Search Sources API
203,203 studies31 sourcesUpdated 24 min ago
Peer-reviewedCC BY 4.0Frontiers in Neurology

Comparative performance of nine large language models on professional Parkinson’s disease tasks: a multidimensional benchmark study

  • Ludi Xie1,2
  • Rumeng Wei2,3
  • Li Xiong2,3
  • Rui Lu1
  • Kangwen Peng2
  • Wenlin Ma1
  1. 1Department of Geriatric Medicine, Tongji Hospital, School of Medicine, Tongji University
  2. 2Department of Neurology, Tongji Hospital, School of Medicine, Tongji University
  3. 3Center for Translational Neurodegeneration and Regenerative Therapy, Tongji Hospital Affiliated to Tongji University School of Medicine
Open at sourceDOICite

Abstract

Background: Parkinson’s disease (PD) requires both disease-specific knowledge and contextual clinical reasoning. However, most evaluations of medical large language models (LLMs) emphasize closed-ended questions and factual accuracy, providing limited insight into broader dimensions of performance on professional PD tasks.

Methods: Nine contemporary LLMs were evaluated using a 200-item English-language professional PD benchmark spanning five clinical domains, four difficulty levels, and multiple question formats. Responses were assessed for expert-rated factual accuracy, clinical reasoning, Global Quality Scale (GQS), and Flesch–Kincaid Grade Level (FKGL).

Results: The nine models generated 1,800 responses. Between-model separation was minimal for expert-rated factual accuracy (Kendall’ s W = 0.031) but greater for FKGL (W = 0.235), Clinical Reasoning (W = 0.256), and GQS (W = 0.301). Between-model separation in expert-rated factual accuracy was small for closed-ended questions (W = 0.020, p = 0.001) but greater for open-ended questions (W = 0.121, p < 0.001). Expert-rated factual accuracy remained weakly discriminative across PD domains and difficulty levels, while GQS showed the greatest domain-specific separation in Safety & Ethics (W = 0.431). Flagship variants consistently showed higher Clinical Reasoning and GQS, but not uniformly higher factual accuracy.

Conclusion: Contemporary LLMs showed limited between-model separation in expert-rated factual accuracy within this benchmark and scoring framework, whereas greater differentiation emerged in open-ended tasks, clinical reasoning, overall response quality, and readability. Multidimensional evaluation may therefore provide a more informative characterization of model performance and help inform task-specific model selection and oversight in medical applications.

Related

  1. Immunosenescence as a modifiable driver of neurodegenerative diseases: Pathways and multidimensional emerging strategies for Alzheimer's and Parkinson's diseases.
    Jingyu Zhao, Linzhuo Qu, Hongjian Guan +3
    Neurotherapeutics : the journal of the American Society for Experimental NeuroTherapeuticsPeer-reviewedCC BY-NC-NDFull textNeurologyEurope PMC
  2. Patient expectations and healthcare professional education in obesity care: A comparative qualitative study
    Tak Ying Louise Ko, Francisca Contreras, Carel W. Le Roux +1
    Obesity pillarsPeer-reviewedCC BYFull textGeneral medicineEurope PMC
  3. Anisotropic conductivity modeling for tDCS in Parkinson’s disease using multidimensional diffusion MRI
    Santiago Osorio Jurado, Mikael Skorpil, Per Svenningsson +2
    Frontiers in Human NeurosciencePeer-reviewedCC BY 4.0Full textNeurologyPMC
  4. Editorial: Health, performance, and injury prevention in professional athletes: a public health approach
    Marko Joksimović, António José Figueiredo, Matej Plevnik +1
    Frontiers in Public HealthPeer-reviewedCC BY 4.0Epidemiology & public healthCrossref
  5. Reliability, Readability, and Safety of Generative AI Chatbots for Osteoporosis Health Counseling: A Multidimensional Cross‐Sectional Comparative Study
    Huimin Xu, Liang Guo, Li Shen +3
    Health science reportsPeer-reviewedCC BY-NC-NDFull textGeneral medicineEurope PMC

Cite

BibTeXRIS
BibTeX
@article{xie2026comparative,
  title = {{Comparative performance of nine large language models on professional Parkinson’s disease tasks: a multidimensional benchmark study}},
  author = {Ludi Xie and Rumeng Wei and Li Xiong and Rui Lu and Kangwen Peng and Wenlin Ma},
  journal = {Frontiers in Neurology},
  year = {2026},
  month = oct,
  doi = {10.3389/fneur.2026.1970459},
  url = {https://www.frontiersin.org/articles/10.3389/fneur.2026.1970459/full},
  note = {CC BY 4.0},
}
RIS
TY  - JOUR
TI  - Comparative performance of nine large language models on professional Parkinson’s disease tasks: a multidimensional benchmark study
AU  - Ludi Xie
AU  - Rumeng Wei
AU  - Li Xiong
AU  - Rui Lu
AU  - Kangwen Peng
AU  - Wenlin Ma
PY  - 2026
DA  - 2026/10/30
JO  - Frontiers in Neurology
DO  - 10.3389/fneur.2026.1970459
UR  - https://www.frontiersin.org/articles/10.3389/fneur.2026.1970459/full
AB  - Background: Parkinson’s disease (PD) requires both disease-specific knowledge and contextual clinical reasoning. However, most evaluations of medical large language models (LLMs) emphasize closed-ended questions and factual accuracy, providing limited insight into broader dimensions of performance on professional PD tasks.  Methods: Nine contemporary LLMs were evaluated using a 200-item English-language professional PD benchmark spanning five clinical domains, four difficulty levels, and multiple question formats. Responses were assessed for expert-rated factual accuracy, clinical reasoning, Global Quality Scale (GQS), and Flesch–Kincaid Grade Level (FKGL).  Results: The nine models generated 1,800 responses. Between-model separation was minimal for expert-rated factual accuracy (Kendall’ s W = 0.031) but greater for FKGL (W = 0.235), Clinical Reasoning (W = 0.256), and GQS (W = 0.301). Between-model separation in expert-rated factual accuracy was small for closed-ended questions (W = 0.020, p = 0.001) but greater for open-ended questions (W = 0.121, p < 0.001). Expert-rated factual accuracy remained weakly discriminative across PD domains and difficulty levels, while GQS showed the greatest domain-specific separation in Safety & Ethics (W = 0.431). Flagship variants consistently showed higher Clinical Reasoning and GQS, but not uniformly higher factual accuracy.  Conclusion: Contemporary LLMs showed limited between-model separation in expert-rated factual accuracy within this benchmark and scoring framework, whereas greater differentiation emerged in open-ended tasks, clinical reasoning, overall response quality, and readability. Multidimensional evaluation may therefore provide a more informative characterization of model performance and help inform task-specific model selection and oversight in medical applications.
SN  - 1664-2295
ER  -