Balancing innovation and ethics: responsible governance of artificial intelligence hallucinations in healthcare.
- 1Law School, Xi'an Jiaotong University, Xi'an, China.
- 2Law School, Xi'an Jiaotong University, Xi'an, China. [email protected].
Abstract
Generative artificial intelligence (AI) is increasingly used in healthcare and may improve efficiency in clinical support, health management, and doctor-patient communication. Yet its tendency to hallucinate, that is, to generate medical information that appears plausible but is false, can create systemic ethical risks in high-risk medical settings. This paper addresses two questions: what distinctive ethical risks are created by medical AI hallucinations, and what governance mechanisms can respond to them? Using the four principles of biomedical ethics as an analytical tool, the paper identifies four risk pathways: weakened informed consent, erosion of doctor-patient trust, harm to life and health, and unfair responsibility allocation. It then introduces responsible research and innovation (RRI) to build a four-dimensional framework of anticipation, inclusion, reflexivity, and responsiveness. The main contribution of the paper is to integrate biomedical ethical principles with RRI governance tools in the specific context of medical AI hallucinations, developing a closed-loop governance matrix across technology development, clinical application, and regulatory oversight. This provides an operational pathway for moving medical AI from technical feasibility towards ethical trustworthiness.
Related
- Editorial: Decoding cancer complexity with artificial intelligence and molecular level studies: informatics-driven approaches for therapeutic innovation
- Responsible Use of Artificial Intelligence Tools in Imaging Reporting: Practical Guidelines for Sonographers-A Narrative Review.
- Ensuring Safe and Responsible Use of Artificial Intelligence in Medical Imaging: Pitfalls and Practical Solutions through the Lens of Obstetric Ultrasound.
- Generative artificial intelligence in the preparation of scientific medical articles on surgery: principles of responsible use and current requirements of editorial organisations
- Regulation of medical artificial intelligence in public health: governance lessons and implications for China
BibTeX
@article{du2026balancing,
title = {{Balancing innovation and ethics: responsible governance of artificial intelligence hallucinations in healthcare.}},
author = {Meixin Du and Ni Zhan},
journal = {Philosophy, ethics, and humanities in medicine : PEHM},
year = {2026},
month = nov,
doi = {10.1186/s13010-026-00237-y},
url = {https://europepmc.org/article/MED/42972332},
note = {CC BY-NC-ND},
}
RIS
TY - JOUR
TI - Balancing innovation and ethics: responsible governance of artificial intelligence hallucinations in healthcare.
AU - Meixin Du
AU - Ni Zhan
PY - 2026
DA - 2026/11/02
JO - Philosophy, ethics, and humanities in medicine : PEHM
DO - 10.1186/s13010-026-00237-y
UR - https://europepmc.org/article/MED/42972332
AB - Generative artificial intelligence (AI) is increasingly used in healthcare and may improve efficiency in clinical support, health management, and doctor-patient communication. Yet its tendency to hallucinate, that is, to generate medical information that appears plausible but is false, can create systemic ethical risks in high-risk medical settings. This paper addresses two questions: what distinctive ethical risks are created by medical AI hallucinations, and what governance mechanisms can respond to them? Using the four principles of biomedical ethics as an analytical tool, the paper identifies four risk pathways: weakened informed consent, erosion of doctor-patient trust, harm to life and health, and unfair responsibility allocation. It then introduces responsible research and innovation (RRI) to build a four-dimensional framework of anticipation, inclusion, reflexivity, and responsiveness. The main contribution of the paper is to integrate biomedical ethical principles with RRI governance tools in the specific context of medical AI hallucinations, developing a closed-loop governance matrix across technology development, clinical application, and regulatory oversight. This provides an operational pathway for moving medical AI from technical feasibility towards ethical trustworthiness.
SN - 1747-5341
ER -