LiteMedCoT-VL: Parameter-Efficient Adaptation for Medical Visual Question Answering
Abstract
The reasoning gap between large and compact vision-language models (VLMs) limits the deployment of medical AI on portable clinical devices. Compact VLMs of 2-4B parameters can run on resource-constrained hardware but lack the multi-step reasoning capacity needed for interpretable clinical decision support. Existing knowledge distillation methods transfer answers without the reasoning process behind them. Medical visual question answering (VQA) serves as a testbed for this problem, as it requires models to integrate visual evidence with clinical knowledge through structured reasoning chains. We introduce LiteMedCoT-VL, a pipeline that transfers chain-of-thought reasoning from a 235B teacher model to 2B student models through LoRA-based fine-tuning on explanation-enriched training data. All inference is conducted without image captions by default, simulating the clinical scenario in which a physician interprets a medical image directly without an accompanying radiology report. On the PMC-VQA benchmark, LiteMedCoT-VL achieves 64.9% accuracy, exceeding the zero-shot Qwen3-VL-4B baseline of 53.9% by 11.0 percentage points and outperforming all published baselines. This result indicates that a 2B model with reasoning distillation can match or exceed models with twice the parameters. Visual grounding analysis shows that the model relies on image content rather than exploiting textual priors. Our code is publicly available at https://github.com/R4nzer/LiteMedCoT-VL.
Related
- MEMOIR-VLM—a multimodal vision-language model for Alzheimer's disease classification and question answering
- Version-aware question answering over living oncology guidelines
- Parameter-Efficient Adaptation of Protein Language Models for Dual-Target Antimicrobial Peptide Discovery Under Strict Homology Constraints
- Relation-aware dynamic knowledge subgraph enhanced large language models for medical multiple-choice question answering
- Entity-centric evaluation of large language model responses for medical question-answering tasks
BibTeX
@misc{ma2026litemedcotvl,
title = {{LiteMedCoT-VL: Parameter-Efficient Adaptation for Medical Visual Question Answering}},
author = {Runze Ma and Shunbo Jia and Haonan Lyu and Guo Liu and Caizhi Liao},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2605.09384},
eprint = {2605.09384},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.09384},
}
RIS
TY - UNPB
TI - LiteMedCoT-VL: Parameter-Efficient Adaptation for Medical Visual Question Answering
AU - Runze Ma
AU - Shunbo Jia
AU - Haonan Lyu
AU - Guo Liu
AU - Caizhi Liao
PY - 2026
DA - 2026/09/18
PB - arXiv
DO - 10.48550/arxiv.2605.09384
UR - https://arxiv.org/abs/2605.09384
AB - The reasoning gap between large and compact vision-language models (VLMs) limits the deployment of medical AI on portable clinical devices. Compact VLMs of 2-4B parameters can run on resource-constrained hardware but lack the multi-step reasoning capacity needed for interpretable clinical decision support. Existing knowledge distillation methods transfer answers without the reasoning process behind them. Medical visual question answering (VQA) serves as a testbed for this problem, as it requires models to integrate visual evidence with clinical knowledge through structured reasoning chains. We introduce LiteMedCoT-VL, a pipeline that transfers chain-of-thought reasoning from a 235B teacher model to 2B student models through LoRA-based fine-tuning on explanation-enriched training data. All inference is conducted without image captions by default, simulating the clinical scenario in which a physician interprets a medical image directly without an accompanying radiology report. On the PMC-VQA benchmark, LiteMedCoT-VL achieves 64.9% accuracy, exceeding the zero-shot Qwen3-VL-4B baseline of 53.9% by 11.0 percentage points and outperforming all published baselines. This result indicates that a 2B model with reasoning distillation can match or exceed models with twice the parameters. Visual grounding analysis shows that the model relies on image content rather than exploiting textual priors. Our code is publicly available at https://github.com/R4nzer/LiteMedCoT-VL.
ER -