Comprehensive Echocardiography Interpretation Using Video and Multiview Vision-Language AI
- 1Department of Cardiovascular Medicine, The University of Tokyo Hospital, Tokyo, Japan
- 2Department of Clinical Laboratory, The University of Tokyo Hospital, Tokyo, Japan
- 3International University of Health and Welfare, Tokyo, Japan
Abstract
Background: Echocardiography is essential for assessing cardiac structure and function, yet accurate interpretation requires specialized expertise, creating challenges in emergency care and in regions with limited access to experienced echocardiographers. Although artificial intelligence–based interpretation is increasingly studied, most existing models rely on still images or single views and do not reflect the video-based, multiview integration used in clinical practice.
Objectives: The authors aim to develop and evaluate a multiview video-language model that integrates cardiac motion and multiple standard echocardiographic views.
Methods: We trained a vision-language model on 577,061 transthoracic echocardiography videos paired with Japanese clinical reports from 46,852 examinations acquired at a single tertiary center (2015-2023). For each examination, features from 5 standard views (parasternal long-axis and short-axis; apical 2-, 3-, and 4-chamber) were aggregated. Performance was assessed in a retrieval task selecting the correct report from 16,436 candidate reports in the test cohort based on video input; the primary metric was correct match retrieval probability within the top 10 candidates (ie, R@10).
Results: The image-based baseline achieved an R@10 of 1.3%. Using video input increased R@10 to 4.2%. Integrating 5 views further improved R@10 to 6.0% (95% CI: 5.5%-6.4%). Gains were greatest for findings that depended on temporal dynamics and multiview assessment, including left ventricular wall motion abnormality and left ventricular dilation.
Conclusions: A multiview video-language framework improved report retrieval compared with conventional image-based approaches. These findings support the utility of video-based, multiview representation learning for echocardiographic report retrieval.
Related
- A clinician aligned vision language framework for stepwise interpretation in fundus fluorescein angiography
- Multi-camera vision-based structural health monitoring of historic masonry minarets with LLM/VLM-assisted damage interpretation
- Automated Interpretation of EEG Reports Using a Large Language Model with Structured Confidence Outputs
- Unifying Vision and Language: Benchmarking End to End Transformer Model Against the ClipCap Framework
- Accuracy Overstates Evidence Grounding and Abstention Reliability in Mammography Vision-Language Models
BibTeX
@article{takizawa2026comprehensive,
title = {{Comprehensive Echocardiography Interpretation Using Video and Multiview Vision-Language AI}},
author = {Ryo Takizawa and Chiemi Yamazaki and Satoshi Kodera and Tempei Kabayama and Shun Kitamura and Risa Kishikawa and Ryo Matsuoka and Megumi Hirokawa and Junichi Ishida and Koki Nakanishi and Hiroyuki Morita and Norihiko Takeda},
journal = {JACC Asia},
year = {2026},
month = aug,
doi = {10.1016/j.jacasi.2026.06.039},
url = {https://doi.org/10.1016/j.jacasi.2026.06.039},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Comprehensive Echocardiography Interpretation Using Video and Multiview Vision-Language AI
AU - Ryo Takizawa
AU - Chiemi Yamazaki
AU - Satoshi Kodera
AU - Tempei Kabayama
AU - Shun Kitamura
AU - Risa Kishikawa
AU - Ryo Matsuoka
AU - Megumi Hirokawa
AU - Junichi Ishida
AU - Koki Nakanishi
AU - Hiroyuki Morita
AU - Norihiko Takeda
PY - 2026
DA - 2026/08/27
JO - JACC Asia
DO - 10.1016/j.jacasi.2026.06.039
UR - https://doi.org/10.1016/j.jacasi.2026.06.039
AB - Background: Echocardiography is essential for assessing cardiac structure and function, yet accurate interpretation requires specialized expertise, creating challenges in emergency care and in regions with limited access to experienced echocardiographers. Although artificial intelligence–based interpretation is increasingly studied, most existing models rely on still images or single views and do not reflect the video-based, multiview integration used in clinical practice. Objectives: The authors aim to develop and evaluate a multiview video-language model that integrates cardiac motion and multiple standard echocardiographic views. Methods: We trained a vision-language model on 577,061 transthoracic echocardiography videos paired with Japanese clinical reports from 46,852 examinations acquired at a single tertiary center (2015-2023). For each examination, features from 5 standard views (parasternal long-axis and short-axis; apical 2-, 3-, and 4-chamber) were aggregated. Performance was assessed in a retrieval task selecting the correct report from 16,436 candidate reports in the test cohort based on video input; the primary metric was correct match retrieval probability within the top 10 candidates (ie, R@10). Results: The image-based baseline achieved an R@10 of 1.3%. Using video input increased R@10 to 4.2%. Integrating 5 views further improved R@10 to 6.0% (95% CI: 5.5%-6.4%). Gains were greatest for findings that depended on temporal dynamics and multiview assessment, including left ventricular wall motion abnormality and left ventricular dilation. Conclusions: A multiview video-language framework improved report retrieval compared with conventional image-based approaches. These findings support the utility of video-based, multiview representation learning for echocardiographic report retrieval.
SN - 2772-3747
ER -