Bridging the experience gap: reasoning-oriented LLMs as cognitive scaffolds for junior anesthesiologists-a multi-arm controlled study.
- 1Department of Anesthesiology, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, China.
- 2Department of Anesthesiology, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, China. [email protected].
Abstract
Background
The gap between junior and senior physicians in anesthesia is exacerbated by the reliance on experience. Large language models like DeepSeek-R1 offer a potential solution as "cognitive scaffolding."Methods
A simulation-based, multi-arm controlled study with randomized junior groups and a non-randomized senior expert comparator was conducted. Forty-eight (48) anesthesiologists, including 16 senior experts and 32 junior residents, were recruited and randomly assigned (using a computer-generated randomization sequence) to two groups: Junior-alone and Junior + AI. Participants evaluated 20 standard clinical scenarios across four clinical domains: airway management, hemodynamic regulation, anesthetic pharmacology, and crisis management. The AI-assisted group followed a triple-step method: independent initial decision, subsequent AI consultation, and decision revision. Their final performance was compared against sixteen (16) senior expert anesthesiologists who served as the gold standard.Results
DeepSeek-R1 successfully improved performance in all four domains for junior doctors. Compared to the Junior-alone group, the Junior + AI group achieved significantly higher scores in accuracy (86.00 ± 16.61 vs 61.67 ± 19.45, P < 0.0001) and comprehensiveness (84.33 ± 17.18 vs 53.67 ± 23.27, P < 0.0001). However, further analysis revealed that while significantly improved, both accuracy and comprehensiveness remained lower than the senior expert scores (differences not statistically significant in mixed-effects analysis: P = 0.349 for Accuracy; P = 0.335 for Comprehensiveness). Performance in safety showed no significant difference to expert standards (P = 0.644). The AI-assisted group required slightly less total time (33.3 vs 33.8 min), showing no significant delay. 65% of AI suggestions were adopted overall.Conclusion
DeepSeek-R1 shows promise as "cognitive scaffolding", supporting knowledge externalization for junior anesthesiologists. It substantially improves decision-making, though a numerical gap with experts persists in certain dimensions. The high risk of automation bias in critical scenarios requires targeted training on critical evaluation of AI outputs.
Related
- Proximity to primary board certification and prior crisis management experience are associated with anesthesiologists performance in high-fidelity acute care scenarios
- Preserving Clinical Reasoning in ChatGPT-Integrated Medical Education: From AI Use to Cognitive Engagement.
- Enhancing Evidence Translation Through Structured Clinical Reasoning: A Mixed-Methods Controlled Feasibility Study of Peer Visibility and Readiness Mechanisms in Applied Learning
- Reclaiming human clinical reasoning in generative AI-enabled healthcare: a perspective on cognitive agency, skill evolution, and accountability
- Clinical expertise and cognitive reflection predict fewer misconceptions in Bayesian diagnostic reasoning
BibTeX
@article{yang2026bridging,
title = {{Bridging the experience gap: reasoning-oriented LLMs as cognitive scaffolds for junior anesthesiologists-a multi-arm controlled study.}},
author = {Huiju Yang and Guangyan Zhang and Zhipeng Wang},
journal = {BMC anesthesiology},
year = {2026},
month = aug,
doi = {10.1186/s12871-026-04173-4},
url = {https://europepmc.org/article/MED/43009790},
note = {CC BY},
}
RIS
TY - JOUR
TI - Bridging the experience gap: reasoning-oriented LLMs as cognitive scaffolds for junior anesthesiologists-a multi-arm controlled study.
AU - Huiju Yang
AU - Guangyan Zhang
AU - Zhipeng Wang
PY - 2026
DA - 2026/08/14
JO - BMC anesthesiology
DO - 10.1186/s12871-026-04173-4
UR - https://europepmc.org/article/MED/43009790
AB - Background The gap between junior and senior physicians in anesthesia is exacerbated by the reliance on experience. Large language models like DeepSeek-R1 offer a potential solution as "cognitive scaffolding."Methods A simulation-based, multi-arm controlled study with randomized junior groups and a non-randomized senior expert comparator was conducted. Forty-eight (48) anesthesiologists, including 16 senior experts and 32 junior residents, were recruited and randomly assigned (using a computer-generated randomization sequence) to two groups: Junior-alone and Junior + AI. Participants evaluated 20 standard clinical scenarios across four clinical domains: airway management, hemodynamic regulation, anesthetic pharmacology, and crisis management. The AI-assisted group followed a triple-step method: independent initial decision, subsequent AI consultation, and decision revision. Their final performance was compared against sixteen (16) senior expert anesthesiologists who served as the gold standard.Results DeepSeek-R1 successfully improved performance in all four domains for junior doctors. Compared to the Junior-alone group, the Junior + AI group achieved significantly higher scores in accuracy (86.00 ± 16.61 vs 61.67 ± 19.45, P < 0.0001) and comprehensiveness (84.33 ± 17.18 vs 53.67 ± 23.27, P < 0.0001). However, further analysis revealed that while significantly improved, both accuracy and comprehensiveness remained lower than the senior expert scores (differences not statistically significant in mixed-effects analysis: P = 0.349 for Accuracy; P = 0.335 for Comprehensiveness). Performance in safety showed no significant difference to expert standards (P = 0.644). The AI-assisted group required slightly less total time (33.3 vs 33.8 min), showing no significant delay. 65% of AI suggestions were adopted overall.Conclusion DeepSeek-R1 shows promise as "cognitive scaffolding", supporting knowledge externalization for junior anesthesiologists. It substantially improves decision-making, though a numerical gap with experts persists in certain dimensions. The high risk of automation bias in critical scenarios requires targeted training on critical evaluation of AI outputs.
SN - 1471-2253
ER -