No significant differences between large language model prompting strategies and human experts in summarisation of 'the formal EU-US meniscus rehabilitation consensus: An ESSKA-AOSSM-AASPT Initiative'.
10 affiliations
- 1Unit of Physiotherapy, Department of Health and Rehabilitation Institute of Neuroscience and Physiology, Sahlgrenska Academy, University of Gothenburg Gothenburg Sweden.
- 2Sahlgrenska Sports Medicine Center, Sahlgrenska Academy Gothenburg Sweden.
- 3Department of Orthopedic Surgery Balgrist University Hospital, University of Zürich Zurich Switzerland.
- 4Department of Computer Science and Engineering Chalmers University of Technology and University of Gothenburg Gothenburg Sweden.
- 5Division of Occupational Therapy and Physical Therapy Cincinnati Children's Hospital and University of Cincinnati College of Medicine Cincinnati Ohio USA.
- 6Department of Physical Therapy University of Delaware Newark Delaware USA.
- 7Institute of Physiotherapy of Saint-Etienne - Saint-Michel Campus Saint- Etienne France.
- 8Center of Orthopaedics and Traumatology, University Hospital Brandenburg, Brandenburg Medical School Theodor Fontane Brandenburg/Havel Germany.
- 9Physiotherapy Research Laboratory University Centre of Physiotherapy and Rehabilitation, Faculty of Physiotherapy, Wroclaw Medical University Wroclaw Poland.
- 10Department of Orthopaedics Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg Gothenburg Sweden.
Abstract
This study aimed to evaluate the interpretive and applicative performance of three conditions applied to the formal EU-US, ESSKA-AOSSM-AASPT consensus on meniscus rehabilitation: (1) single-pass prompting, (2) sequential chain-of-thought prompting and (3) human expert-written output. This exploratory comparative study evaluated three conditions applied to two parts of the formal EU-US ESSKA-AOSSM-AASPT meniscus rehabilitation consensus: (1) single-pass prompting, (2) sequential chain-of-thought prompting using ChatGPT (GPT-5.1), Claude (Sonnet 4.5), and Gemini (3.0), and (3) human expert-written output from three collaborating human experts. For each part, three tasks were assessed: generation of summaries, application of summaries to three predefined clinical scenarios, and resistance to three predefined misinterpretation queries. Three blinded senior sports physiotherapists evaluated summary quality using five QUEST categories and rated the clinical application and misinterpretation resistance responses. Median ratings were compared using Friedman and post-hoc Wilcoxon signed-rank tests. Gwet's AC2 coefficient with quadratic weights was used to assess inter-rater agreement for each QUEST category and for the clinical application and misinterpretation resistance items. Across consensus parts, no statistically significant differences were identified between large language model prompting strategies and human expert outputs for QUEST categories, clinical application, and misinterpretation resistance queries. Sequential chain-of-thought prompting received higher numerical reviewer ratings than single-pass prompting for ChatGPT and Gemini, particularly for quality of information and trust and confidence, although differences were not statistically significant. Inter-rater agreement ranged from substantial to almost perfect (Gwet's AC2 = 0.79-0.99). No statistically significant differences in reviewer ratings were observed between ChatGPT, Claude, Gemini and human expert output, with generally high ratings for summaries and responses to clinical scenarios. Large language models may therefore represent a potential adjunct for summarisation and interpretation of sports medicine consensus recommendations to inform rehabilitation decision-making. Level V.
Related
- Impact evaluation of Lean Healthcare in emergency departments: evidence from a large-scale Brazilian initiative using difference-in-differences
- Age differences in motivations, barriers, and facilitators of wearable and mobile app use in a government-led digital health initiative: a qualitative study
- Elderly men and women with similar areal bone density show significant differences in bone microstructure and fracture pattern.
- Strategies to establish a biologically defined Parkinson's disease cohort prior to the onset of clinically significant motor symptoms
- Integrative ALS therapy: gene targeted therapy by base editing and potential exercise rehabilitation as synergistic strategies for functional recovery
BibTeX
@article{lindskog2026significant,
title = {{No significant differences between large language model prompting strategies and human experts in summarisation of 'the formal EU-US meniscus rehabilitation consensus: An ESSKA-AOSSM-AASPT Initiative'.}},
author = {Jakob Lindskog and Felix Öttl and Yinan Yu and Mark V Paterno and Airelle Giordano and Alexandre Rambaud and Robert Prill and Aleksandra Królikowska and Kristian Samuelsson and Eric Hamrin Senorski},
journal = {Journal of experimental orthopaedics},
year = {2026},
month = oct,
doi = {10.1002/jeo2.70964},
url = {https://europepmc.org/article/MED/42953767},
note = {CC BY},
}
RIS
TY - JOUR
TI - No significant differences between large language model prompting strategies and human experts in summarisation of 'the formal EU-US meniscus rehabilitation consensus: An ESSKA-AOSSM-AASPT Initiative'.
AU - Jakob Lindskog
AU - Felix Öttl
AU - Yinan Yu
AU - Mark V Paterno
AU - Airelle Giordano
AU - Alexandre Rambaud
AU - Robert Prill
AU - Aleksandra Królikowska
AU - Kristian Samuelsson
AU - Eric Hamrin Senorski
PY - 2026
DA - 2026/10/28
JO - Journal of experimental orthopaedics
DO - 10.1002/jeo2.70964
UR - https://europepmc.org/article/MED/42953767
AB - This study aimed to evaluate the interpretive and applicative performance of three conditions applied to the formal EU-US, ESSKA-AOSSM-AASPT consensus on meniscus rehabilitation: (1) single-pass prompting, (2) sequential chain-of-thought prompting and (3) human expert-written output. This exploratory comparative study evaluated three conditions applied to two parts of the formal EU-US ESSKA-AOSSM-AASPT meniscus rehabilitation consensus: (1) single-pass prompting, (2) sequential chain-of-thought prompting using ChatGPT (GPT-5.1), Claude (Sonnet 4.5), and Gemini (3.0), and (3) human expert-written output from three collaborating human experts. For each part, three tasks were assessed: generation of summaries, application of summaries to three predefined clinical scenarios, and resistance to three predefined misinterpretation queries. Three blinded senior sports physiotherapists evaluated summary quality using five QUEST categories and rated the clinical application and misinterpretation resistance responses. Median ratings were compared using Friedman and post-hoc Wilcoxon signed-rank tests. Gwet's AC2 coefficient with quadratic weights was used to assess inter-rater agreement for each QUEST category and for the clinical application and misinterpretation resistance items. Across consensus parts, no statistically significant differences were identified between large language model prompting strategies and human expert outputs for QUEST categories, clinical application, and misinterpretation resistance queries. Sequential chain-of-thought prompting received higher numerical reviewer ratings than single-pass prompting for ChatGPT and Gemini, particularly for quality of information and trust and confidence, although differences were not statistically significant. Inter-rater agreement ranged from substantial to almost perfect (Gwet's AC2 = 0.79-0.99). No statistically significant differences in reviewer ratings were observed between ChatGPT, Claude, Gemini and human expert output, with generally high ratings for summaries and responses to clinical scenarios. Large language models may therefore represent a potential adjunct for summarisation and interpretation of sports medicine consensus recommendations to inform rehabilitation decision-making. Level V.
SN - 2197-1153
ER -