Scoring Rubric to Assess Ethical Issues in Clinical Artificial Intelligence Tools
- 1University of Pittsbugh
Abstract
With the growing use of artificial intelligence (AI) tools in clinical care, it is essential to assess ethical issues related to using these tools, both before and after deployment. To address this need, a multidisciplinary team of clinical and AI experts developed and tested a scoring rubric to evaluate AI tools in clinical settings from an ethical perspective, considering potential issues that may arise during their use. The rubric provides a replicable scoring system comprising five content areas and four scoring categories. The content areas include evaluation population, accessibility, clinical bias, user control, and monitoring and updates. To test the applicability of the rubric, we applied it to three AI tools designed for clinical decision-making, clinical documentation, and clinical image analysis. The evaluation relied solely on publicly available information to simulate typical clinical settings. The rubric addresses a critical gap in the evaluation and governance of clinical AI tools by providing a structured framework to assess their ethical aspects. This approach enables healthcare systems to address and monitor the ethical impact of AI tools, fostering greater trust among clinicians and patients and guiding improvements in clinical practice.
Related
- Preliminary Clinical Evaluation in Cross‐Sectional Imaging: A Qualitative Inquiry Into the Ethical Considerations in the Era of Artificial Intelligence
- Clinical Validation, Bias, and Ethical Deployment of Artificial Intelligence in Imaging.
- Introducing Artificial Intelligence into Surgical Practice for Cancer Care: Clinical Applications and Ethical Considerations
- Test-Retest Reliability of Large Language Models and Their Concordance With Human Experts in Rubric-Based Scoring of Clinical Summary Statements: Experimental Comparative Study.
- Knowledge, Awareness, and Perception toward Ethical Guidelines for Artificial Intelligence in Biomedical Research and Healthcare among Dental Faculty and Postgraduates – A Cross-sectional Study
BibTeX
@misc{kindler2026scoring,
title = {{Scoring Rubric to Assess Ethical Issues in Clinical Artificial Intelligence Tools}},
author = {K. Kindler and S. Visweswaran and Y. Wang and J. Abad and J. Maier and O. V. Kravchenko},
howpublished = {medRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.05.26362340},
url = {https://www.medrxiv.org/content/10.64898/2026.09.05.26362340v1},
note = {CC BY-ND},
}
RIS
TY - UNPB
TI - Scoring Rubric to Assess Ethical Issues in Clinical Artificial Intelligence Tools
AU - K. Kindler
AU - S. Visweswaran
AU - Y. Wang
AU - J. Abad
AU - J. Maier
AU - O. V. Kravchenko
PY - 2026
DA - 2026/09/05
PB - medRxiv
DO - 10.64898/2026.09.05.26362340
UR - https://www.medrxiv.org/content/10.64898/2026.09.05.26362340v1
AB - With the growing use of artificial intelligence (AI) tools in clinical care, it is essential to assess ethical issues related to using these tools, both before and after deployment. To address this need, a multidisciplinary team of clinical and AI experts developed and tested a scoring rubric to evaluate AI tools in clinical settings from an ethical perspective, considering potential issues that may arise during their use. The rubric provides a replicable scoring system comprising five content areas and four scoring categories. The content areas include evaluation population, accessibility, clinical bias, user control, and monitoring and updates. To test the applicability of the rubric, we applied it to three AI tools designed for clinical decision-making, clinical documentation, and clinical image analysis. The evaluation relied solely on publicly available information to simulate typical clinical settings. The rubric addresses a critical gap in the evaluation and governance of clinical AI tools by providing a structured framework to assess their ethical aspects. This approach enables healthcare systems to address and monitor the ethical impact of AI tools, fostering greater trust among clinicians and patients and guiding improvements in clinical practice.
ER -