{
  "abstract": "PurposeOpen-source vision-language models (VLMs) can be locally deployed without external internet access. This study aimed to evaluate the slice-level lesion-detection performance of multiple general-purpose and medical-purpose open-source VLMs for brain metastases on contrast-enhanced (CE) MRI, and to assess their accuracy in characterizing detailed imaging features on lesion-positive images.\n\nMaterials and MethodsOne hundred lesion-positive axial CE T1-weighted images and 100 matched lesion-negative images from 100 patients were analyzed using eight evaluable VLMs: five general-purpose VLMs-InternVL3-8B, Qwen2.5-VL-7B-Instruct, MiniCPM-V-4.5, DeepSeek-VL2-tiny, and Phi-3.5-vision-instruct-and three medical-purpose VLMs-MedGemma-4B-it, LLaVA-Med v1.5, and HuatuoGPT-Vision-7B. Lesion-detection performance was assessed using sensitivity, specificity, and balanced accuracy. On lesion-positive images, accuracy was evaluated for lesion count, laterality, anatomic location, enhancement pattern, necrosis, and mass effect. Balanced accuracy was reported with patient-level cluster bootstrap 95% confidence intervals (CIs). Models were compared using the Friedman test and paired patient-level permutation tests with Benjamini-Hochberg correction. Secondary outcomes were compared with majority-class baselines using exact McNemar tests. A revised prompt was tested post hoc.\n\nResultsThe median age of the study patients was 65 years (IQR, 59.0-70.0), and 58 patients were male (58.0%). MiniCPM-V-4.5 showed significantly higher balanced accuracy (80.5%; 95% CI, 75.0-85.5) than all other models (FDR-adjusted P < 0.001). Seven of the eight models showed predicted-positive rates of 81-100%, and 70.8% (17/24) of responses to synthetic control images without anatomic content were classified as lesion-positive. With the revised prompt, balanced accuracy increased from 59.5% to 77.0% in HuatuoGPT-Vision-7B and from 51.5% to 68.5% in InternVL3-8B. No model significantly outperformed the majority-class baseline for enhancement pattern or for mass effect.\n\nConclusionThis slice-level technical benchmark demonstrated variable lesion-detection performance of open-source VLMs for brain metastasis in an image-only setting, with most models showing a strong positive-response tendency.",
  "abstract_shown": true,
  "affiliations": [
    "Department of Radiology, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea"
  ],
  "article_type": "PUBLISHAHEADOFPRINT",
  "arxiv": null,
  "author_total": 7,
  "authors": [
    {
      "affs": [
        0
      ],
      "name": "J. Kim"
    },
    {
      "affs": [
        0
      ],
      "name": "B.-s. Kim"
    },
    {
      "name": "J. S. Ko"
    },
    {
      "name": "J. Dong"
    },
    {
      "name": "S. Y. Youn"
    },
    {
      "name": "J. Jang"
    },
    {
      "name": "K.-J. Ahn"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/7z4rp8y253ygn.bib",
  "cited_by": 0,
  "doi": "10.64898/2026.08.24.26361169",
  "fulltext_url": "https://www.medrxiv.org/content/10.64898/2026.08.24.26361169v2.full",
  "id": "7z4rp8y253ygn",
  "issns": [],
  "landing_url": "https://www.medrxiv.org/content/10.64898/2026.08.24.26361169v2",
  "language": null,
  "licence": "CC-BY-NC-ND",
  "licence_label": "CC BY-NC-ND",
  "licence_source": "article",
  "licence_url": "https://creativecommons.org/licenses/by-nc-nd/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "medrxiv",
      "radiology and imaging"
    ],
    [
      "openalex",
      "2718|Artificial Intelligence in Healthcare and Education"
    ]
  ],
  "openalex": "W7204258682",
  "pdf_url": "https://www.medrxiv.org/content/10.64898/2026.08.24.26361169v2.full.pdf",
  "pmcid": null,
  "pmid": null,
  "published": "2026-08-24",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/7z4rp8y253ygn.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:20:20Z",
      "native_id": "10.64898/2026.08.24.26361169",
      "source": "medrxiv",
      "url": "https://www.medrxiv.org/content/10.64898/2026.08.24.26361169v2"
    }
  ],
  "subjects": [
    {
      "field": "medicine",
      "origin": "medrxiv:radiology and imaging",
      "subfield": "imaging"
    }
  ],
  "title": "Benchmarking Open-Source Vision-Language Models for Brain Metastasis Assessment on Single-Slice Contrast-Enhanced MRI",
  "title_text": "Benchmarking Open-Source Vision-Language Models for Brain Metastasis Assessment on Single-Slice Contrast-Enhanced MRI",
  "type": "preprint",
  "updated_at": "2026-10-30T12:20:20Z",
  "url": "https://research.draconitic.com/paper/7z4rp8y253ygn",
  "venue": "medRxiv",
  "versions": []
}