Explainable AI in Healthcare: A Holistic View of Technical Approaches, Regulatory Frameworks, Reliable Clinical Implementation Insights and Limitations.
5 affiliations
- 1Instituto de Ciência e Inovação em Engenharia Mecânica e Engenharia Industrial, Campus of the Faculty of Engineering, Universidade do Porto (FEUP), R. Dr. Roberto Frias 400, 4200-465 Porto, Portugal.
- 2BioGHP, Rua da Constituição, Nº 656, 1º, Sala 117, 4200-194 Porto, Portugal.
- 3Faculty of Medicine, Universidade do Porto (FMUP), Alameda Prof. Hernâni Monteiro, 4200-319 Porto, Portugal.
- 4World Gastroenterology Organisation (WGO) Porto Gastroenterology and Hepatology Training Center, Campus of Faculty of Medicine, Universidade do Porto (FMUP), Alameda Prof. Hernâni Monteiro, 4200-319 Porto, Portugal.
- 5Department of Gastroenterology, São João University Hospital, Alameda Prof. Hernâni Monteiro, 4200-319 Porto, Portugal.
Abstract
Opaque deep learning models in biomedical applications pose challenges related to bias, fairness, and regulatory compliance. This paper presents a narrative review of explainable artificial intelligence (XAI) in healthcare, specifically medical diagnosis, based on a targeted selection guided by a narrative review of recent literature, from 2019 to 2026, conducted on Scopus, Web of Science, and PubMed databases. Covered topics comprise model interpretability, including image-based models and multimodal models, with special focus on large language models (LLMs) as a hard-to-interpret system. The review situates XAI within key regulatory frameworks, including the European Union (EU) Artificial Intelligence (AI) Act, highlighting how explainability supports legal requirements for transparency, auditability, and accountability. It examines widely used XAI methods such as Shapley Additive Explanations (SHAP), Local Interpretable Model-Agnostic Explanations (LIME), attention visualization, neurosymbolic reasoning, and others. These techniques help identify bias, detect spurious correlations, and analyze hallucinations in LLM deployment. They present an effort to, within a feasible range, minimize these events and increase confidence in the predictions provided, without dispensing with clear and proper validation within the medical classification loop. While emphasizing the benefits of XAI, the review also addresses key limitations, including limited explanation fidelity, cognitive overload, and the risk of misleading interpretations. Overall, it provides a concise synthesis of how explainability intersects with ethics, regulation, and system design to support oversight and transparency in AI healthcare. Additionally, it provides suggestions for the incorporation of technical XAI methods within current legislation and guidelines to provide a robust framework for AI model deployment.
Related
- Advances in Diagnostic Testing for Eosinophilic Esophagitis: Current Approaches, Limitations, and Emerging Technologies
- Social prescribing for anxiety and depression in older adults: mechanistic pathways, implementation requirements, and limitations of current evidence
- Endoscopic Endonasal Approaches to Anterior Cranial Base Meningiomas: Indications, Limitations, and Surgical Decision-Making.
- Data approaches for lifecourse research on early-life determinants of late-life health outcomes: opportunities, limitations, and tradeoffs
- A modified Delphi study to determine Australian healthcare consumer priorities and consensus for the use of genomics and precision medicine approaches in pregnancy healthcare and research
BibTeX
@article{barbosa2026explainable,
title = {{Explainable AI in Healthcare: A Holistic View of Technical Approaches, Regulatory Frameworks, Reliable Clinical Implementation Insights and Limitations.}},
author = {Guilherme Barbosa and Eduardo Carvalho and Dulce Oliveira and Ana Rita Moura and Ana Guerra and Francisca Pinheiro and Ricardo Correia and André Martins Dos Santos and Nilza Ramião and Miguel Mascarenhas},
journal = {Diagnostics (Basel, Switzerland)},
year = {2026},
month = sep,
doi = {10.3390/diagnostics16193107},
url = {https://europepmc.org/article/MED/42882263},
note = {CC BY},
}
RIS
TY - JOUR
TI - Explainable AI in Healthcare: A Holistic View of Technical Approaches, Regulatory Frameworks, Reliable Clinical Implementation Insights and Limitations.
AU - Guilherme Barbosa
AU - Eduardo Carvalho
AU - Dulce Oliveira
AU - Ana Rita Moura
AU - Ana Guerra
AU - Francisca Pinheiro
AU - Ricardo Correia
AU - André Martins Dos Santos
AU - Nilza Ramião
AU - Miguel Mascarenhas
PY - 2026
DA - 2026/09/24
JO - Diagnostics (Basel, Switzerland)
DO - 10.3390/diagnostics16193107
UR - https://europepmc.org/article/MED/42882263
AB - Opaque deep learning models in biomedical applications pose challenges related to bias, fairness, and regulatory compliance. This paper presents a narrative review of explainable artificial intelligence (XAI) in healthcare, specifically medical diagnosis, based on a targeted selection guided by a narrative review of recent literature, from 2019 to 2026, conducted on Scopus, Web of Science, and PubMed databases. Covered topics comprise model interpretability, including image-based models and multimodal models, with special focus on large language models (LLMs) as a hard-to-interpret system. The review situates XAI within key regulatory frameworks, including the European Union (EU) Artificial Intelligence (AI) Act, highlighting how explainability supports legal requirements for transparency, auditability, and accountability. It examines widely used XAI methods such as Shapley Additive Explanations (SHAP), Local Interpretable Model-Agnostic Explanations (LIME), attention visualization, neurosymbolic reasoning, and others. These techniques help identify bias, detect spurious correlations, and analyze hallucinations in LLM deployment. They present an effort to, within a feasible range, minimize these events and increase confidence in the predictions provided, without dispensing with clear and proper validation within the medical classification loop. While emphasizing the benefits of XAI, the review also addresses key limitations, including limited explanation fidelity, cognitive overload, and the risk of misleading interpretations. Overall, it provides a concise synthesis of how explainability intersects with ethics, regulation, and system design to support oversight and transparency in AI healthcare. Additionally, it provides suggestions for the incorporation of technical XAI methods within current legislation and guidelines to provide a robust framework for AI model deployment.
SN - 2075-4418
ER -