A clinician aligned vision language framework for stepwise interpretation in fundus fluorescein angiography
- 1Zhejiang University, Eye Center of Second Affiliated Hospital, School of Medicine, China. Zhejiang Provincial Key Laboratory of Ophthalmology. Zhejiang Provincial Clinical Research Center for Eye Diseases. Zhejiang Provincial Engineering Institute on Eye Diseases, Zhejiang University, Hangzhou, China
- 2State Key Laboratory of Transvascular Implantation Devices and TIDRI, School of Medicine, Zhejiang University, Hangzhou, China
- 3Zhejiang University School of Medicine, Hangzhou, China
Abstract
Fluorescein fundus angiography (FFA) is essential for diagnosing retinal vascular diseases, yet its interpretation is expertise-intensive. Here, we present Clin-FFA-VLM, a multimodal vision–language framework that mirrors retina specialists’ cognitive workflow by decomposing FFA interpretation into three stages: lesion-aware visual perception, clinical report generation, and diagnostic decision support. Trained and tested on a multi-center dataset of 13,178 FFA images with expert label, 21,717 FFA images with 1790 clinical reports and diagnosis across 7 retinal diseases, Clin-FFA-VLM achieves an F1 of 0.834 for lesion detection, an entity-level F1 of 0.73 for report generation, and a diagnostic F1 of 0.77 by jointly reasoning over images and self-generated reports. External validation across two independent hospitals confirmed its generalizability (F1 of 0.78 and 0.70). In a prospective reader study with 200 FFA cases, Clin-FFA-VLM significantly improved diagnostic accuracy for medical students and residents (p < 0.05), bridging the gap between automated systems and clinical practice.
Related
- Classification of cystoid macular edema status from retinal vasculitis fluorescein angiography images using a convolutional neural network.
- Longitudinal Quantitative Analysis of Ultra-Widefield Fluorescein Angiography Leakage Metrics: Analysis of the KINGFISHER Clinical Trial
- ClinStigma: A Clinician-Guided Framework for Developing Synthetic Benchmarks of Contextual Stigmatizing Language in Clinical Documentation
- Using Vision and Large Language Models for Psychological Intervention Competency Assessment: A Pilot Study of Agreement with Clinician Ratings
- Automated Interpretation of EEG Reports Using a Large Language Model with Structured Confidence Outputs
BibTeX
@article{su2026clinician,
title = {{A clinician aligned vision language framework for stepwise interpretation in fundus fluorescein angiography}},
author = {Zichang Su and Xiaocong Liu and Bingtao Guan and An Shao and Xindi Liu and Yan Yan and Ziyao Luo and Zhikang Li and Yufeng Xu and Jian Wu and Xiaoling Huang and Juan Ye},
journal = {NPJ Digital Medicine},
year = {2026},
month = jul,
doi = {10.1038/s41746-026-02932-8},
url = {https://doi.org/10.1038/s41746-026-02932-8},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - JOUR
TI - A clinician aligned vision language framework for stepwise interpretation in fundus fluorescein angiography
AU - Zichang Su
AU - Xiaocong Liu
AU - Bingtao Guan
AU - An Shao
AU - Xindi Liu
AU - Yan Yan
AU - Ziyao Luo
AU - Zhikang Li
AU - Yufeng Xu
AU - Jian Wu
AU - Xiaoling Huang
AU - Juan Ye
PY - 2026
DA - 2026/07/12
JO - NPJ Digital Medicine
DO - 10.1038/s41746-026-02932-8
UR - https://doi.org/10.1038/s41746-026-02932-8
AB - Fluorescein fundus angiography (FFA) is essential for diagnosing retinal vascular diseases, yet its interpretation is expertise-intensive. Here, we present Clin-FFA-VLM, a multimodal vision–language framework that mirrors retina specialists’ cognitive workflow by decomposing FFA interpretation into three stages: lesion-aware visual perception, clinical report generation, and diagnostic decision support. Trained and tested on a multi-center dataset of 13,178 FFA images with expert label, 21,717 FFA images with 1790 clinical reports and diagnosis across 7 retinal diseases, Clin-FFA-VLM achieves an F1 of 0.834 for lesion detection, an entity-level F1 of 0.73 for report generation, and a diagnostic F1 of 0.77 by jointly reasoning over images and self-generated reports. External validation across two independent hospitals confirmed its generalizability (F1 of 0.78 and 0.70). In a prospective reader study with 200 FFA cases, Clin-FFA-VLM significantly improved diagnostic accuracy for medical students and residents (p < 0.05), bridging the gap between automated systems and clinical practice.
SN - 2398-6352
ER -