A competency-based assessment system for undergraduate oral and maxillofacial surgery internship: development and pilot validation.
- 1Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, No.22, Zhongguancun South Street, Beijing, China. [email protected].
- 2Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, No.22, Zhongguancun South Street, Beijing, China.
- 3Department of General Dentistry II, Peking University School and Hospital of Stomatology, No.22, Zhongguancun South Street, Beijing, China.
Abstract
Background
Traditional assessment in undergraduate oral and maxillofacial surgery (OMFS) internships mostly relies on single-skill examinations, focusing only on technical proficiency while ignoring communication ability, clinical reasoning and medical documentation. This study aimed to develop and pilot-validate a comprehensive competency-based assessment system for undergraduate OMFS outpatient internships.Methods
This mixed-methods development and pilot validation study identified six core clinical competency domains through literature review and semi-structured interviews with five senior OMFS faculty members. A multi-dimensional assessment framework was established, including theoretical examination, integrated clinical operation evaluation, phased skill assessment aligned with national licensing standards, patient-related indicators and daily performance evaluation. Forty-six students in the development cohort (2024-2025) adopted the new system; 48 students in the reference cohort (2023-2024) received traditional assessment. Each student was independently evaluated by three faculty members from an eleven-person rater panel. Inter-rater reliability was calculated using the intraclass correlation coefficient. Questionnaire feedback and adverse event data were compared between groups.Results
Six core competency domains were finalized. Student ratings of the comprehensiveness of the two assessment systems were not significantly different (8.14 vs. 6.52, P = 0.086). Faculty ratings showed a significant difference in comprehensiveness (9.64 vs. 6.28, P = 0.043). The overall inter-rater reliability was good (ICC = 0.78, 95% CI: 0.65-0.87). Based on participant feedback, the weight of theoretical examination and patient satisfaction was reduced, and more objective indicators were added. No significant between-group differences were found in adverse event incidence.Trial registration
Not applicable.
Related
- Designing and validating a clinical competency logbook for workplace-based assessment in undergraduate medical education: A foundational psychometric analysis
- A journey mapping study of internship stress among undergraduate nursing interns
- Assessing cultural awareness and competency in nursing education: A cross-sectional study of undergraduate nursing students in Ireland
- Defining competency frameworks for undergraduate and graduate pharmacology education in Canada
- Assessment in competency-based medical education for anesthesiology: validity evidence, equity, and faculty readiness
BibTeX
@article{zhang2026competencybased,
title = {{A competency-based assessment system for undergraduate oral and maxillofacial surgery internship: development and pilot validation.}},
author = {Yan-Yan Zhang and Shuang Yang and Wen-Bo Zhang and Na An and Nian-Hui Cui},
journal = {BMC medical education},
year = {2026},
month = jul,
doi = {10.1186/s12909-026-10026-y},
url = {https://europepmc.org/article/MED/42902218},
note = {CC BY-NC-ND},
}
RIS
TY - JOUR
TI - A competency-based assessment system for undergraduate oral and maxillofacial surgery internship: development and pilot validation.
AU - Yan-Yan Zhang
AU - Shuang Yang
AU - Wen-Bo Zhang
AU - Na An
AU - Nian-Hui Cui
PY - 2026
DA - 2026/07/01
JO - BMC medical education
DO - 10.1186/s12909-026-10026-y
UR - https://europepmc.org/article/MED/42902218
AB - Background Traditional assessment in undergraduate oral and maxillofacial surgery (OMFS) internships mostly relies on single-skill examinations, focusing only on technical proficiency while ignoring communication ability, clinical reasoning and medical documentation. This study aimed to develop and pilot-validate a comprehensive competency-based assessment system for undergraduate OMFS outpatient internships.Methods This mixed-methods development and pilot validation study identified six core clinical competency domains through literature review and semi-structured interviews with five senior OMFS faculty members. A multi-dimensional assessment framework was established, including theoretical examination, integrated clinical operation evaluation, phased skill assessment aligned with national licensing standards, patient-related indicators and daily performance evaluation. Forty-six students in the development cohort (2024-2025) adopted the new system; 48 students in the reference cohort (2023-2024) received traditional assessment. Each student was independently evaluated by three faculty members from an eleven-person rater panel. Inter-rater reliability was calculated using the intraclass correlation coefficient. Questionnaire feedback and adverse event data were compared between groups.Results Six core competency domains were finalized. Student ratings of the comprehensiveness of the two assessment systems were not significantly different (8.14 vs. 6.52, P = 0.086). Faculty ratings showed a significant difference in comprehensiveness (9.64 vs. 6.28, P = 0.043). The overall inter-rater reliability was good (ICC = 0.78, 95% CI: 0.65-0.87). Based on participant feedback, the weight of theoretical examination and patient satisfaction was reduced, and more objective indicators were added. No significant between-group differences were found in adverse event incidence.Trial registration Not applicable.
SN - 1472-6920
ER -