Objective assessment of surgical performance in otolaryngology residency.
- 1Department of Otolaryngology - Head and Neck Surgery, Hotel Dieu de France Hospital, Saint Joseph University, P.O.Box: 166830, Alfred Naccache Boulevard, Ashrafieh, Beirut, Lebanon. [email protected].
- 2Department of Otolaryngology - Head and Neck Surgery, Hotel Dieu de France Hospital, Saint Joseph University, P.O.Box: 166830, Alfred Naccache Boulevard, Ashrafieh, Beirut, Lebanon.
Abstract
Purpose
A surgeon's technical competence is a key predictor of surgical outcomes, yet residents' performance is often assessed by direct observation without structured, predefined criteria, which can be biased and unreliable. This study aimed to implement structured assessment tools across multiple procedures in an ENT residency, assess their feasibility, and evaluate their impact on resident performance.Methods
Two procedures per residency year were selected. Seven validated tools were adopted from the literature and an eighth was newly developed; each comprised a Task-Specific Checklist (TSC) and a Global Rating Scale (GRS). Residents were assessed over two rounds, each by three physicians plus a self-evaluation. The primary outcomes were implementation of the eight grids and the feasibility of multi-evaluator, multi-procedure assessment in routine practice. The secondary outcome was impact on performance measured as the change in TSC and GRS scores between rounds and the faculty-self agreement.Results
All eight grids were implemented and digitized; assessment was feasible for six of eight procedures, as endoscopic sinus surgery and pediatric bronchoscopy yielded too few eligible cases. No significant between-round differences were found for individual procedures, though non-significant improvement trends were observed. Pooled self-assessments improved significantly over time. Self- and faculty assessments did not differ significantly, though limited statistical power precludes firm conclusions. Junior residents improved most, whereas senior residents showed a ceiling effect.Conclusion
This pilot study demonstrates the feasibility of structured, multi-evaluator surgical assessment in an ENT residency. Using multiple evaluators limits rater bias and will enable inter-rater reliability testing, not feasible at this stage. These preliminary results support a larger, multi-center study incorporating reliability and validity testing before broader implementation.
Related
- An Objective Assessment for Plank Performance in Active Adults With and Without Low Back Pain: A Reliability and Comparison Study
- Artificial Intelligence–Based Surgical Phase Analysis Enables Objective Assessment of Surgeon Skill in Robotic Distal Gastrectomy: A Multicenter Study
- Surgical Performance and Patient Outcomes in Robot-Assisted Esophagectomy: A Crowd-Sourced Video-Based Assessment
- Assessment of Perioperative Nurses’ Performance Regarding Surgical Site Infection Prevention Bundle Application
- Toward a Predictive Model for Medical Residency Entrance Examination Performance in Brazil: A Preregistered Cross-Sectional Study and Exploratory Prototype Score Calculator
BibTeX
@article{melkane2026objective,
title = {{Objective assessment of surgical performance in otolaryngology residency.}},
author = {Antoine E Melkane and Nadim Khoueir and Simon Rassi},
journal = {BMC medical education},
year = {2026},
month = jul,
doi = {10.1186/s12909-026-09991-1},
url = {https://europepmc.org/article/MED/42902331},
note = {CC BY-NC-ND},
}
RIS
TY - JOUR
TI - Objective assessment of surgical performance in otolaryngology residency.
AU - Antoine E Melkane
AU - Nadim Khoueir
AU - Simon Rassi
PY - 2026
DA - 2026/07/01
JO - BMC medical education
DO - 10.1186/s12909-026-09991-1
UR - https://europepmc.org/article/MED/42902331
AB - Purpose A surgeon's technical competence is a key predictor of surgical outcomes, yet residents' performance is often assessed by direct observation without structured, predefined criteria, which can be biased and unreliable. This study aimed to implement structured assessment tools across multiple procedures in an ENT residency, assess their feasibility, and evaluate their impact on resident performance.Methods Two procedures per residency year were selected. Seven validated tools were adopted from the literature and an eighth was newly developed; each comprised a Task-Specific Checklist (TSC) and a Global Rating Scale (GRS). Residents were assessed over two rounds, each by three physicians plus a self-evaluation. The primary outcomes were implementation of the eight grids and the feasibility of multi-evaluator, multi-procedure assessment in routine practice. The secondary outcome was impact on performance measured as the change in TSC and GRS scores between rounds and the faculty-self agreement.Results All eight grids were implemented and digitized; assessment was feasible for six of eight procedures, as endoscopic sinus surgery and pediatric bronchoscopy yielded too few eligible cases. No significant between-round differences were found for individual procedures, though non-significant improvement trends were observed. Pooled self-assessments improved significantly over time. Self- and faculty assessments did not differ significantly, though limited statistical power precludes firm conclusions. Junior residents improved most, whereas senior residents showed a ceiling effect.Conclusion This pilot study demonstrates the feasibility of structured, multi-evaluator surgical assessment in an ENT residency. Using multiple evaluators limits rater bias and will enable inter-rater reliability testing, not feasible at this stage. These preliminary results support a larger, multi-center study incorporating reliability and validity testing before broader implementation.
SN - 1472-6920
ER -