Interpretable agentic AI system with localized reasoning for radiology
7 affiliations
- 1Department of Electrical Engineering, City University of Hong Kong, Hong Kong, SAR China
- 2Department of Radiology, Massachusetts General Hospital, Boston, MA USA
- 3Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, SAR China
- 4School of Computing, University of Georgia, Athens, GA USA
- 5Harvard University, Boston, MA USA
- 6Department of Computer Science and Engineering, Lehigh University, Bethlehem, PA USA
- 7Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, SAR China
Abstract
Medical AI has produced many radiology models, particularly for chest X-rays (CXR), each excelling at isolated tasks like lesion detection or report generation. However, these models have disparate capabilities and limited generalizability due to training on restricted datasets, making clinical integration challenging. Large language models (LLMs) now enable interfacing heterogeneous models within agentic frameworks that automatically interpret and unify outputs in natural language. In this work, we present RadFabric, an agentic AI system that orchestrates fourteen specialized open-source CXR analytics models and two Vision-Language Models (VLM) through a modular protocol. RadFabric includes an Anatomical Interpretation Agent that grounds visual findings in anatomical context, and a trainable reasoning agent that synthesizes these anatomically-enriched outputs with VLM-generated radiology reports into transparent, step-by-step diagnoses, even when model outputs are heterogeneous or conflicting. This architecture enables explainable, robust diagnoses across common and rare pathologies while facilitating extensibility through additional agents. Evaluation results on the MIMIC-CXR dataset shows that RadFabric can achieve an AUC of 85.18% on task of detecting different legion types from the given CXR, outperforming all state-of-art CXR models. Notably, the reasoning agent particularly improves detection of uncommon findings, demonstrating enhanced interpretability, generalizability, and clinical applicability.
Related
- Artificial Intelligence in MSK Radiology: Current Applications, LLMs, Agentic AI, and Future Directions.
- The Radiology Sparring Partner: A PoC Study of Multi-Agentic Workflows for Clinical Decision Support and Teaching
- Skill-guided agentic reasoning enables longitudinal synthesis of myeloma records at expert-level concordance.
- Can Jev Judge Radiology Reports? Evaluating a System One Model for Clinical Factuality
- From Clinical Free Text to Auditable Concepts: An Agentic Framework for Interpretable Prediction
BibTeX
@article{chen2026interpretable,
title = {{Interpretable agentic AI system with localized reasoning for radiology}},
author = {Wenting Chen and Yi Dong and Zhaojun Ding and Yucheng Shi and Fang Zeng and Yifan Zhou and Huaqin Zhao and Yijun Luo and Tianyu Lin and Yihang Su and Yichen Wu and Jasmine Liu and Kai Zhang and Wenxuan Wang and Zhen Xiang and Tianming Liu and Ninghao Liu and Quanzheng Li and Lichao Sun and Yixuan Yuan and Xiang Li},
journal = {NPJ Digital Medicine},
year = {2026},
month = jul,
doi = {10.1038/s41746-026-02994-8},
url = {https://doi.org/10.1038/s41746-026-02994-8},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - JOUR
TI - Interpretable agentic AI system with localized reasoning for radiology
AU - Wenting Chen
AU - Yi Dong
AU - Zhaojun Ding
AU - Yucheng Shi
AU - Fang Zeng
AU - Yifan Zhou
AU - Huaqin Zhao
AU - Yijun Luo
AU - Tianyu Lin
AU - Yihang Su
AU - Yichen Wu
AU - Jasmine Liu
AU - Kai Zhang
AU - Wenxuan Wang
AU - Zhen Xiang
AU - Tianming Liu
AU - Ninghao Liu
AU - Quanzheng Li
AU - Lichao Sun
AU - Yixuan Yuan
AU - Xiang Li
PY - 2026
DA - 2026/07/15
JO - NPJ Digital Medicine
DO - 10.1038/s41746-026-02994-8
UR - https://doi.org/10.1038/s41746-026-02994-8
AB - Medical AI has produced many radiology models, particularly for chest X-rays (CXR), each excelling at isolated tasks like lesion detection or report generation. However, these models have disparate capabilities and limited generalizability due to training on restricted datasets, making clinical integration challenging. Large language models (LLMs) now enable interfacing heterogeneous models within agentic frameworks that automatically interpret and unify outputs in natural language. In this work, we present RadFabric, an agentic AI system that orchestrates fourteen specialized open-source CXR analytics models and two Vision-Language Models (VLM) through a modular protocol. RadFabric includes an Anatomical Interpretation Agent that grounds visual findings in anatomical context, and a trainable reasoning agent that synthesizes these anatomically-enriched outputs with VLM-generated radiology reports into transparent, step-by-step diagnoses, even when model outputs are heterogeneous or conflicting. This architecture enables explainable, robust diagnoses across common and rare pathologies while facilitating extensibility through additional agents. Evaluation results on the MIMIC-CXR dataset shows that RadFabric can achieve an AUC of 85.18% on task of detecting different legion types from the given CXR, outperforming all state-of-art CXR models. Notably, the reasoning agent particularly improves detection of uncommon findings, demonstrating enhanced interpretability, generalizability, and clinical applicability.
SN - 2398-6352
ER -