Development and validation of a digital learning resource quality evaluation framework: a mixed-methods study in health professions education
- 1School of Public Health, Kunming Medical University
- 2Academy of Biomedical Engineering, Kunming Medical University
Abstract
Background: Digital learning resources (DLRs) are now core teaching tools in health professions education, yet quality evaluation remains constrained by subjective weighting bias and a persistent gap between educational importance and operational measurability. Because DLRs increasingly mediate what and how students learn, deficiencies in resource quality propagate directly into competency attainment, making quality evaluation a proximal determinant of student competency outcomes. This study aimed to develop and validate a practical DLR quality evaluation framework for medical and public health education.
Methods: A sequential mixed-methods design was employed. First, a modified Delphi procedure with 26 experts was used to construct the hierarchical indicator system. Second, an expert-based and data-derived integrated weighting model combining AHP and EWM was developed to balance expert judgment and data discriminatory power. Third, a measurability correction mechanism and a four-quadrant prioritization matrix were established. The framework was validated with 10 DLRs across 10 medical courses, involving 309 students.
Results: The final system comprised 4 primary dimensions, 9 secondary indicators, and 14 tertiary indicators. Content Quality had the highest integrated weight (0.469), with Data Visualization (0.136) and Updated Content and Frontier Knowledge (0.103) as the top tertiary indicators. The four-quadrant matrix classified 7 indicators for direct implementation and 2 for targeted methodological development. Validation confirmed high internal consistency (Cronbach’s α = 0.987) and satisfactory content validity (S-CVI = 0.951); absolute-agreement inter-rater ICCs were near zero, indicating substantial between-student variability in scoring levels.
Conclusion: This study provides a reliable, implementable DLR quality evaluation framework. By converting evaluation results into ranked, measurable improvement priorities, the framework directs resource revision toward the attributes that bear most directly on student competency development. The methodological workflow shows potential for transferability across nursing, pharmacy, public health and other health-related disciplines via contextual calibration, pending formal cross-context validation.
Related
- Symbolic ableism in health professions education: development and initial psychometric evaluation of the German symbolic ableism scale.
- Adapting the Girandole framework for cross-disciplinary social justice reflection in health professions education
- Enhancing patient understanding of ovarian cancer: creation, validation and user-readability evaluation of an education booklet
- Who Learns from Disruption? Antifragility and Burden Transfer in Health Professions Education
- Beyond the clicker: rethinking student response systems as pedagogical infrastructure for healthcare professions education
BibTeX
@article{cui2026development,
title = {{Development and validation of a digital learning resource quality evaluation framework: a mixed-methods study in health professions education}},
author = {Wenlong Cui and Yuan Zhang and Wenlin Dong and Yang Xu and Fang Xu and Tiantian Zhao and Yan Zhang and Lingli Dong and Hua Bai and Dingyun You and Huixin Wu},
journal = {Frontiers in Public Health},
year = {2026},
month = oct,
doi = {10.3389/fpubh.2026.1958724},
url = {https://www.frontiersin.org/articles/10.3389/fpubh.2026.1958724/full},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Development and validation of a digital learning resource quality evaluation framework: a mixed-methods study in health professions education
AU - Wenlong Cui
AU - Yuan Zhang
AU - Wenlin Dong
AU - Yang Xu
AU - Fang Xu
AU - Tiantian Zhao
AU - Yan Zhang
AU - Lingli Dong
AU - Hua Bai
AU - Dingyun You
AU - Huixin Wu
PY - 2026
DA - 2026/10/14
JO - Frontiers in Public Health
DO - 10.3389/fpubh.2026.1958724
UR - https://www.frontiersin.org/articles/10.3389/fpubh.2026.1958724/full
AB - Background: Digital learning resources (DLRs) are now core teaching tools in health professions education, yet quality evaluation remains constrained by subjective weighting bias and a persistent gap between educational importance and operational measurability. Because DLRs increasingly mediate what and how students learn, deficiencies in resource quality propagate directly into competency attainment, making quality evaluation a proximal determinant of student competency outcomes. This study aimed to develop and validate a practical DLR quality evaluation framework for medical and public health education. Methods: A sequential mixed-methods design was employed. First, a modified Delphi procedure with 26 experts was used to construct the hierarchical indicator system. Second, an expert-based and data-derived integrated weighting model combining AHP and EWM was developed to balance expert judgment and data discriminatory power. Third, a measurability correction mechanism and a four-quadrant prioritization matrix were established. The framework was validated with 10 DLRs across 10 medical courses, involving 309 students. Results: The final system comprised 4 primary dimensions, 9 secondary indicators, and 14 tertiary indicators. Content Quality had the highest integrated weight (0.469), with Data Visualization (0.136) and Updated Content and Frontier Knowledge (0.103) as the top tertiary indicators. The four-quadrant matrix classified 7 indicators for direct implementation and 2 for targeted methodological development. Validation confirmed high internal consistency (Cronbach’s α = 0.987) and satisfactory content validity (S-CVI = 0.951); absolute-agreement inter-rater ICCs were near zero, indicating substantial between-student variability in scoring levels. Conclusion: This study provides a reliable, implementable DLR quality evaluation framework. By converting evaluation results into ranked, measurable improvement priorities, the framework directs resource revision toward the attributes that bear most directly on student competency development. The methodological workflow shows potential for transferability across nursing, pharmacy, public health and other health-related disciplines via contextual calibration, pending formal cross-context validation.
SN - 2296-2565
ER -