Potential risks of generative artificial intelligence in kidney care
5 affiliations
- 1Nephrology Department, Fundació Puigvert, Barcelona, Spain.
- 2Institut de Recerca Sant Pau (IR SANT PAU), Barcelona, Spain.
- 3Departament de Medicina, Universitat Autònoma de Barcelona, Barcelona, Spain.
- 4RICORS2040KIDNEY, Spain.
- 5IOMED, Barcelona, Spain.
Abstract
Background
Generative artificial intelligence (GenAI), particularly large language models, is rapidly emerging as a transformative tool in medicine, including nephrology, with applications spanning clinical decision support, documentation, patient education, and research. Unlike traditional machine learning systems, GenAI produces probabilistic, context-dependent outputs, introducing novel opportunities but also distinct risks.Methods
This review examines key risk domains associated with the integration of GenAI into kidney care. These include data-related challenges such as bias, limited representativeness, and data quality issues; technical limitations such as hallucinations and prompt-dependent variability; and cognitive risks, including automation bias and overreliance. The differences between human clinical reasoning and AI-generated outputs further complicate safe implementation, particularly in high-stakes decision-making contexts. In addition, broader systemic concerns are addressed, including inequities in access, linguistic and infrastructural disparities, and the environmental footprint associated with large-scale AI systems.Results
We highlight that these risks are not isolated but interconnected, with potential implications for patient safety, clinical decision-making, and health system equity. While GenAI may enhance efficiency and support clinical workflows, its outputs require critical appraisal and validation within the clinical context.Conclusions
Safe and responsible implementation will depend on a combination of technical safeguards, structured prompting, rigorous validation, and human-in-the-loop oversight. Regulatory frameworks provide an essential foundation, but clinical accountability remains central.Ultimately, the integration of GenAI in nephrology should prioritize safety, equity, and sustainability, ensuring that technological innovation translates into meaningful and responsible improvements in patient care.
Related
- Psychometric applications of generative artificial intelligence: Lifecycle, risks, and research agenda
- Generative Artificial Intelligence Models in Breast Cancer Diagnosis: A Systematic Review
- Nursing students’ experiences with using generative artificial intelligence: a descriptive qualitative study
- Generative Artificial Intelligence in Periodontology: Application of OpenAI Models for the Classification of Periodontitis and Risk Assessment in Patients Undergoing Supportive Periodontal Care
- Simulating Psychotherapeutic Sessions with Generative Artificial Intelligence: A Proof-of-Concept Study of In Silico Psychotherapy Research
BibTeX
@article{ramirez2026potential,
title = {{Potential risks of generative artificial intelligence in kidney care}},
author = {Elizabeth R Viera Ramírez and Leonor Fayos de Arizón and Roser Torra and Gabriel Maeztu},
journal = {Clinical kidney journal},
year = {2026},
month = jul,
doi = {10.1093/ckj/sfag200},
url = {https://europepmc.org/article/PMC/PMC13716678},
note = {CC BY-NC},
}
RIS
TY - JOUR
TI - Potential risks of generative artificial intelligence in kidney care
AU - Elizabeth R Viera Ramírez
AU - Leonor Fayos de Arizón
AU - Roser Torra
AU - Gabriel Maeztu
PY - 2026
DA - 2026/07/17
JO - Clinical kidney journal
DO - 10.1093/ckj/sfag200
UR - https://europepmc.org/article/PMC/PMC13716678
AB - Background Generative artificial intelligence (GenAI), particularly large language models, is rapidly emerging as a transformative tool in medicine, including nephrology, with applications spanning clinical decision support, documentation, patient education, and research. Unlike traditional machine learning systems, GenAI produces probabilistic, context-dependent outputs, introducing novel opportunities but also distinct risks.Methods This review examines key risk domains associated with the integration of GenAI into kidney care. These include data-related challenges such as bias, limited representativeness, and data quality issues; technical limitations such as hallucinations and prompt-dependent variability; and cognitive risks, including automation bias and overreliance. The differences between human clinical reasoning and AI-generated outputs further complicate safe implementation, particularly in high-stakes decision-making contexts. In addition, broader systemic concerns are addressed, including inequities in access, linguistic and infrastructural disparities, and the environmental footprint associated with large-scale AI systems.Results We highlight that these risks are not isolated but interconnected, with potential implications for patient safety, clinical decision-making, and health system equity. While GenAI may enhance efficiency and support clinical workflows, its outputs require critical appraisal and validation within the clinical context.Conclusions Safe and responsible implementation will depend on a combination of technical safeguards, structured prompting, rigorous validation, and human-in-the-loop oversight. Regulatory frameworks provide an essential foundation, but clinical accountability remains central.Ultimately, the integration of GenAI in nephrology should prioritize safety, equity, and sustainability, ensuring that technological innovation translates into meaningful and responsible improvements in patient care.
SN - 2048-8505
SN - 2048-8513
ER -