{
  "abstract": "Complex medical reasoning requires integrating heterogeneous clinical evidence across multiple inference steps. Large language models (LLMs) now approach this through two routes: internalized reasoning and externalized agent scaffolding (frameworks that decompose problems collaboratively among multiple LLMs). To determine whether these routes are exclusive or complementary, we introduce MedicalAgentsBench, a filtered benchmark of 862 complex clinical questions drawn from the union of eight medical datasets via difficulty-aware curation and contamination screening. Evaluating three internalized reasoning models (DeepSeek-R1, o1-mini, and o3-mini), seven base models, and nine externalized agent-based methods, we find that internalized and externalized approaches each independently improve performance and that their benefits compound: the highest accuracy is achieved by layering agent workflows onto an internalized reasoning model (i.e., o3-mini + MDAgents with 35.1%). Pareto analysis shows this combination dominates the cost-performance frontier; moreover, lightweight optimization on inexpensive models offers an entry point for resource-constrained settings.",
  "abstract_shown": true,
  "affiliations": [
    "Department of Biomedical Informatics & Data Science, Yale University, New Haven, CT 06520, USA",
    "Program in Computational Biology & Biomedical Informatics, Yale University, New Haven, CT 06520, USA",
    "Department of Computer Science, Yale University, New Haven, CT 06520, USA",
    "Department of Biosystems Science and Engineering, ETH Zurich, 4058 Basel, Switzerland",
    "Department of Computer Science, Stanford University, Stanford, CA 94305, USA",
    "Department of Bioinformatics, UT Southwestern Medical Center, Dallas, TX 75390, USA",
    "Department of Molecular Biophysics and Biochemistry, Yale University, New Haven, CT 06520, USA",
    "Department of Statistics & Data Science, Yale University, New Haven, CT 06520, USA"
  ],
  "article_type": "research article",
  "arxiv": null,
  "author_total": 13,
  "authors": [
    {
      "affs": [
        0,
        1
      ],
      "name": "Yanjun Shao",
      "orcid": "0009-0005-3486-467X"
    },
    {
      "affs": [
        2
      ],
      "name": "Xiangru Tang",
      "orcid": "0009-0006-2700-4513"
    },
    {
      "affs": [
        3
      ],
      "name": "Jiwoong Sohn",
      "orcid": "0009-0001-7962-5833"
    },
    {
      "affs": [
        0,
        1
      ],
      "name": "Jiapeng Chen"
    },
    {
      "affs": [
        0
      ],
      "name": "Yuxuan Liao"
    },
    {
      "affs": [
        2
      ],
      "name": "Jiayi Zhang",
      "orcid": "0000-0002-7334-4256"
    },
    {
      "affs": [
        2
      ],
      "name": "J Xiang"
    },
    {
      "affs": [
        4
      ],
      "name": "Fang Wu",
      "orcid": "0000-0001-7240-3915"
    },
    {
      "affs": [
        2
      ],
      "name": "Yilun Zhao"
    },
    {
      "affs": [
        2
      ],
      "name": "Chenglin Wu",
      "orcid": "0000-0001-7733-1084"
    },
    {
      "affs": [
        5
      ],
      "name": "Wenqi Shi",
      "orcid": "0000-0001-8972-7342"
    },
    {
      "affs": [
        2
      ],
      "name": "Arman Cohan",
      "orcid": "0000-0002-8954-2724"
    },
    {
      "affs": [
        0,
        2,
        6,
        7,
        1
      ],
      "name": "Mark Gerstein",
      "orcid": "0000-0002-9746-3719"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/7t53xmzb8h43k.bib",
  "cited_by": 61,
  "doi": "10.1016/j.patter.2026.101610",
  "fulltext_url": "https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13674309/?tool=EBI",
  "id": "7t53xmzb8h43k",
  "issns": [
    "2666-3899"
  ],
  "landing_url": "https://europepmc.org/article/PMC/PMC13674309",
  "language": "eng",
  "licence": "CC-BY",
  "licence_label": "CC BY",
  "licence_source": "article",
  "licence_url": "https://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "openalex",
      "2718|Artificial Intelligence in Healthcare and Education"
    ]
  ],
  "openalex": "W4415286464",
  "pdf_url": "https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13674309/pdf/?tool=EBI",
  "pmcid": "PMC13674309",
  "pmid": "42746258",
  "published": "2026-07-08",
  "publisher": "Elsevier BV",
  "ris": "https://research.draconitic.com/paper/7t53xmzb8h43k.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:18:30Z",
      "native_id": "PMC:PMC13674309",
      "source": "europepmc",
      "url": "https://europepmc.org/article/PMC/PMC13674309"
    },
    {
      "first_seen": "2026-11-04T13:21:13Z",
      "native_id": "W4415286464",
      "source": "openalex",
      "url": "https://doi.org/10.1016/j.patter.2026.101658"
    }
  ],
  "subjects": [
    {
      "field": "medicine",
      "origin": "openalex:2718|Artificial Intelligence in Healthcare and Education",
      "subfield": "epidemiology-public-health"
    }
  ],
  "title": "MedicalAgentsBench for complex medical reasoning: Comparing internalized reasoning models versus externalized agent-based frameworks",
  "title_text": "MedicalAgentsBench for complex medical reasoning: Comparing internalized reasoning models versus externalized agent-based frameworks",
  "type": "peer-reviewed",
  "updated_at": "2026-11-09T13:21:15Z",
  "url": "https://research.draconitic.com/paper/7t53xmzb8h43k",
  "venue": "Patterns (New York, N.Y.)",
  "versions": []
}