Effectiveness of the "Level-Points" online training model and influencing factors in enhancing scoring accuracy among surgical examiners.
6 affiliations
- 1Clinical Skills Training Center, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, 310009, China.
- 2Department of Epidemiology & Health Statistics, School of Public Health, School of Medicine, Zhejiang University, Hangzhou, 310058, China.
- 3Education Department, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, 310009, China.
- 4Clinical Skills Training Center, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, 310009, China. [email protected].
- 5Clinical Skills Training Center, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, 310009, China. [email protected].
- 6Department of Epidemiology & Health Statistics, School of Public Health, School of Medicine, Zhejiang University, Hangzhou, 310058, China. [email protected].
Abstract
Background
Ensuring a standardized level of scoring capability among surgical examiners is crucial for the fairness and validity of competency-based assessments in standardized residency training. However, traditional in-person training models face significant challenges in scalability and accessibility. The aim of this study was to evaluate the "Level-Points" online training model for enhancing scoring accuracy among surgical examiners and identify its key influencing factors.Methods
This study included data from the 2024 national surgical examiner training program. The training adopted a model integrating level-based progression and a point-reward system, designed in accordance with the national surgical resident competency assessment standards and covering five dimensions. Both before and after the training, the examiners were required to rate the five dimensions in the clinical scenario videos, with scoring deviation and improvement values as the outcomes.Results
A total of 670 surgical examiners were included. The mean dimensional scoring deviation decreased 3.82 points from 19.34 (pre-assessment) to 15.52 (post-assessment) (P < 0.001, adjusted P < 0.001), with statistically significant improvements consistently observed across all five dimensions (All adjusted P < 0.001). Effect sizes ranged from - 0.41 to -1.15 (Cohen's d), with the overall effect size being large (d = -1.09). For baseline scoring deviations, significant associations were found only with examiners' specialty background after FDR correction. For training effectiveness, none of the variables showed statistically significant differences after FDR correction. In dimension-specific analyses, four associations remained statistically significant: plastic surgery in the patient encounter (β = -3.345, se = 1.065, P = 0.002, adjusted P = 0.012) and specialty-specific clinical reasoning (β = 3.915, se = 0.896, P < 0.001, adjusted P < 0.001) dimensions, cardiothoracic surgery in the patient encounter dimension (β = -1.996, se = 0.834, P = 0.017, adjusted P = 0.047), and neurosurgery in the communication skills dimension (β = 4.400, se = 0.933, P < 0.001, adjusted P < 0.001). Professional title and geographic region exhibited moderating effects in specific dimensions but did not survive FDR correction. No significant impact on improvement was observed for age or gender (all adjusted P > 0.05).Conclusion
The "Level-Points" online training model effectively enhanced scoring accuracy among surgical examiners, demonstrating significant scalability advantages. Specialty background emerged as a key factor influencing both baseline scoring deviation and dimension-specific improvements, highlighting the need for refined, discipline-sensitive calibration strategies within standardized assessment frameworks.
Related
- Evaluating the effectiveness of the SPICES model in enhancing students' learning experience in social medicine and public health field training at jiroft university of medical sciences, Iran.
- Evaluation of Voriconazole Population Pharmacokinetics in Japanese Patients and Factors Influencing Prediction Accuracy: A Machine Learning Approach
- Analysis of key performance indicators influencing offensive effectiveness during clutch moments in elite international men’s basketball
- Enhancing deep sleep with auditory stimulation to boost cognitive training responsiveness in people with Parkinson's disease and mild cognitive impairment: Rationale and design of the double-blind randomized controlled trial PD-CogT-Sleep
- Enhancing diagnostic accuracy with core needle biopsy and immunohistochemistry in indeterminate thyroid nodules: a retrospective diagnostic accuracy study.
BibTeX
@article{zhang2026effectiveness,
title = {{Effectiveness of the "Level-Points" online training model and influencing factors in enhancing scoring accuracy among surgical examiners.}},
author = {Lina Zhang and Mengling Yan and Yiwen Qiu and Haolei Cai and Yunxian Yu},
journal = {BMC medical education},
year = {2026},
month = jul,
doi = {10.1186/s12909-026-10033-z},
url = {https://europepmc.org/article/MED/42928189},
note = {CC BY-NC-ND},
}
RIS
TY - JOUR
TI - Effectiveness of the "Level-Points" online training model and influencing factors in enhancing scoring accuracy among surgical examiners.
AU - Lina Zhang
AU - Mengling Yan
AU - Yiwen Qiu
AU - Haolei Cai
AU - Yunxian Yu
PY - 2026
DA - 2026/07/28
JO - BMC medical education
DO - 10.1186/s12909-026-10033-z
UR - https://europepmc.org/article/MED/42928189
AB - Background Ensuring a standardized level of scoring capability among surgical examiners is crucial for the fairness and validity of competency-based assessments in standardized residency training. However, traditional in-person training models face significant challenges in scalability and accessibility. The aim of this study was to evaluate the "Level-Points" online training model for enhancing scoring accuracy among surgical examiners and identify its key influencing factors.Methods This study included data from the 2024 national surgical examiner training program. The training adopted a model integrating level-based progression and a point-reward system, designed in accordance with the national surgical resident competency assessment standards and covering five dimensions. Both before and after the training, the examiners were required to rate the five dimensions in the clinical scenario videos, with scoring deviation and improvement values as the outcomes.Results A total of 670 surgical examiners were included. The mean dimensional scoring deviation decreased 3.82 points from 19.34 (pre-assessment) to 15.52 (post-assessment) (P < 0.001, adjusted P < 0.001), with statistically significant improvements consistently observed across all five dimensions (All adjusted P < 0.001). Effect sizes ranged from - 0.41 to -1.15 (Cohen's d), with the overall effect size being large (d = -1.09). For baseline scoring deviations, significant associations were found only with examiners' specialty background after FDR correction. For training effectiveness, none of the variables showed statistically significant differences after FDR correction. In dimension-specific analyses, four associations remained statistically significant: plastic surgery in the patient encounter (β = -3.345, se = 1.065, P = 0.002, adjusted P = 0.012) and specialty-specific clinical reasoning (β = 3.915, se = 0.896, P < 0.001, adjusted P < 0.001) dimensions, cardiothoracic surgery in the patient encounter dimension (β = -1.996, se = 0.834, P = 0.017, adjusted P = 0.047), and neurosurgery in the communication skills dimension (β = 4.400, se = 0.933, P < 0.001, adjusted P < 0.001). Professional title and geographic region exhibited moderating effects in specific dimensions but did not survive FDR correction. No significant impact on improvement was observed for age or gender (all adjusted P > 0.05).Conclusion The "Level-Points" online training model effectively enhanced scoring accuracy among surgical examiners, demonstrating significant scalability advantages. Specialty background emerged as a key factor influencing both baseline scoring deviation and dimension-specific improvements, highlighting the need for refined, discipline-sensitive calibration strategies within standardized assessment frameworks.
SN - 1472-6920
ER -