TongueBench: performance of prompt-engineered large multimodal models in Traditional Chinese Medicine tongue diagnosis
5 affiliations
- 1Institute of TCM Diagnostics, Hunan University of Chinese Medicine, Changsha, Hunan, China
- 2Provincial Key Laboratory of TCM Diagnostics, Hunan University of Chinese Medicine, Changsha, Hunan, China
- 3School of Software Engineering, Sun Yat-sen University, Zhuhai, China
- 4Department of Computer Science, Cornell University, Ithaca, NY, United States
- 5Academy of Chinese Medical Sciences, Hunan University of Chinese Medicine, Changsha, Hunan, China
Abstract
Introduction: Traditional Chinese Medicine (TCM) tongue diagnosis requires reconstruction of multiple visual signs across tongue morphology, body color, coating texture, and coating color. We developed TongueBench as a prompt-controlled, ontology-constrained evaluation protocol for application programming interface (API)-accessed large multimodal models (LMMs), without task-specific training.
Methods: Nine LMMs were evaluated on 2,694 tongue images with 22 expert-adjudicated labels under direct prompting (DP), knowledge-guided prompting (KGP), and knowledge-guided reflective prompting (KGRP). For the primary LMM-versus-supervised comparison, archived LMM predictions were matched to the same 270 test images used by three supervised baselines. Sensitivity analyses assessed a fixed label order, training-only frequencies, and no conflict removal. Raw-output auditing distinguished model-generated ontology violations from their removal by deterministic post-processing.
Results: Across the nine evaluated models, KGRP increased mean Micro-F1 from 47.57% to 48.51%, Macro-F1 from 32.41% to 33.60%, and sample-level Jaccard from 34.45% to 35.32%, while reducing Hamming loss from 26.39% to 25.54%. Model-level changes were heterogeneous. Image-level bootstrap intervals described the fixed nine-model mean contrast under an unverified image-independence assumption. Supervised models had better observed aggregate scores on the matched split, but patient-level separation could not be established. Sensitivity analyses retained the direction of the mean KGRP-versus-DP changes.
Discussion: TongueBench characterizes modest prompt-associated changes, parsing failures, repeated-output variation, and incomplete label-set reconstruction. These observations do not establish general prompt superiority, patient-level generalizability, or clinical utility.
Related
- Multimodal imaging and artificial intelligence for bone tumor diagnosis: Comparative performance and emerging technologies
- Multimodal Imaging for Cervical Aggressive Fibromatosis: Diagnosis and Individualized Interventional Therapy
- Ultrasound-based multimodal diagnosis and endovascular therapy for phlegmasia cerulea dolens secondary to May–Thurner syndrome in a middle-aged male patient : A case report
- Multimodal Multiple Instance Learning for Fibroepithelial Tumor Diagnosis in Breast Ultrasound
- Risk prediction models for macrovascular complications in type 2 diabetes mellitus: from traditional risk scores to multimodal intelligent prediction
BibTeX
@article{meng2026tonguebench,
title = {{TongueBench: performance of prompt-engineered large multimodal models in Traditional Chinese Medicine tongue diagnosis}},
author = {Yihan Meng and Nuan Cui and Jingxin Yao and Shufeng Kong and Pei Liu and Zengbo Zhang and Hao Liang},
journal = {Frontiers in Artificial Intelligence},
year = {2026},
month = oct,
doi = {10.3389/frai.2026.1943004},
url = {https://doi.org/10.3389/frai.2026.1943004},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - TongueBench: performance of prompt-engineered large multimodal models in Traditional Chinese Medicine tongue diagnosis
AU - Yihan Meng
AU - Nuan Cui
AU - Jingxin Yao
AU - Shufeng Kong
AU - Pei Liu
AU - Zengbo Zhang
AU - Hao Liang
PY - 2026
DA - 2026/10/21
JO - Frontiers in Artificial Intelligence
DO - 10.3389/frai.2026.1943004
UR - https://doi.org/10.3389/frai.2026.1943004
AB - Introduction: Traditional Chinese Medicine (TCM) tongue diagnosis requires reconstruction of multiple visual signs across tongue morphology, body color, coating texture, and coating color. We developed TongueBench as a prompt-controlled, ontology-constrained evaluation protocol for application programming interface (API)-accessed large multimodal models (LMMs), without task-specific training. Methods: Nine LMMs were evaluated on 2,694 tongue images with 22 expert-adjudicated labels under direct prompting (DP), knowledge-guided prompting (KGP), and knowledge-guided reflective prompting (KGRP). For the primary LMM-versus-supervised comparison, archived LMM predictions were matched to the same 270 test images used by three supervised baselines. Sensitivity analyses assessed a fixed label order, training-only frequencies, and no conflict removal. Raw-output auditing distinguished model-generated ontology violations from their removal by deterministic post-processing. Results: Across the nine evaluated models, KGRP increased mean Micro-F1 from 47.57% to 48.51%, Macro-F1 from 32.41% to 33.60%, and sample-level Jaccard from 34.45% to 35.32%, while reducing Hamming loss from 26.39% to 25.54%. Model-level changes were heterogeneous. Image-level bootstrap intervals described the fixed nine-model mean contrast under an unverified image-independence assumption. Supervised models had better observed aggregate scores on the matched split, but patient-level separation could not be established. Sensitivity analyses retained the direction of the mean KGRP-versus-DP changes. Discussion: TongueBench characterizes modest prompt-associated changes, parsing failures, repeated-output variation, and incomplete label-set reconstruction. These observations do not establish general prompt superiority, patient-level generalizability, or clinical utility.
SN - 2624-8212
ER -