Addressing benchmarking gaps in large language models for health and medicine with dynamic red-teaming
11 affiliations
- 1Department of Engineering Science, University of Oxford, Oxford, UK
- 2Munich Center for Machine Learning, Munich, Germany
- 3School of Computation, Information and Technology, Technical University of Munich, Munich, Germany
- 4School of Medicine, TUM University Hospital, Munich, Germany
- 5Department of Computing, Imperial College London, London, UK
- 6Department of Diagnostic Radiology, National University of Singapore, Singapore, Singapore
- 7Department of Computer Science, Stony Brook University, Stony Brook, NY USA
- 8School of Computer Science, University of Sheffield, Sheffield, UK
- 9School of Medicine and Health, Technical University of Munich, Munich, Germany
- 10Department of Neurosurgery/Neuro-Oncology, State Key Laboratory of Oncology in South China, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-sen University Cancer Center, Guangzhou, China
- 11Department of Diagnostic and Interventional Neuroradiology, TUM University Hospital, Munich, Germany
Abstract
Large language models (LLMs) are increasingly used to answer health-related questions and support healthcare workflows, yet evidence for their safety still relies heavily on static benchmarks that can rapidly become obsolete or be optimized against. Here we introduce a Dynamic, Automatic and Systematic (DAS) red-teaming audit framework that continuously stress-tests LLMs for health across four safety-critical axes: robustness, privacy, bias and hallucination. Validated against board-certified clinicians, a suite of adversarial agents autonomously mutates health-related test cases to uncover vulnerabilities in real time. Applying DAS to 15 state-of-the-art LLMs revealed a profound gap between high static benchmark performance and low dynamic reliability—the ‘benchmarking gap’. Despite median MedQA accuracy exceeding 80%, 94% of previously correct answers failed under dynamic robustness testing. This brittleness generalized to the realistic, open-ended HealthBench dataset, where top-tier models exhibited failure rates exceeding 70%, suggesting that high scores on established static benchmarks may reflect superficial memorization. We observed similarly high failure rates across other domains: privacy leaks were elicited in 86% of scenarios, cognitive bias priming altered recommendations in 81% of fairness tests and hallucination rates exceeded 74% in widely used models. By converting LLM safety evaluation for health from a static checklist into a living adversarial audit, DAS provides a scalable framework for surfacing latent risks before such systems are deployed in consumer-facing health assistants and broader clinical workflows.
Related
- Unifying Vision and Language: Benchmarking End to End Transformer Model Against the ClipCap Framework
- Benchmarking modular skills and fixed orchestration across six large language models in a meta-analysis task
- The potential role of large language models in addressing phenoconversion in pharmacogenomics
- BRIDGE: benchmarking large language models for understanding real-world clinical practice texts
- Evaluating large language models in patient education: a comparative analysis addressing frequently asked questions in peri-acetabular osteotomy
BibTeX
@article{pan2026addressing,
title = {{Addressing benchmarking gaps in large language models for health and medicine with dynamic red-teaming}},
author = {Jiazhen Pan and Bailiang Jian and Paul Hager and Yundi Zhang and Che Liu and Friedrike Jungmann and Hongwei Li and Julian Canisius and Chenyu You and Junde Wu and Jiayuan Zhu and Fenglin Liu and Yuyuan Liu and Niklas Bubeck and Moritz A. Knolle and Chen Chen and Christian Wachinger and Zhenyu Gong and Cheng Ouyang and Georgios A. Kaissis and Benedikt Wiestler and Daniel Rueckert},
journal = {Nature Health},
year = {2026},
month = jul,
doi = {10.1038/s44360-026-00152-8},
url = {https://doi.org/10.1038/s44360-026-00152-8},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Addressing benchmarking gaps in large language models for health and medicine with dynamic red-teaming
AU - Jiazhen Pan
AU - Bailiang Jian
AU - Paul Hager
AU - Yundi Zhang
AU - Che Liu
AU - Friedrike Jungmann
AU - Hongwei Li
AU - Julian Canisius
AU - Chenyu You
AU - Junde Wu
AU - Jiayuan Zhu
AU - Fenglin Liu
AU - Yuyuan Liu
AU - Niklas Bubeck
AU - Moritz A. Knolle
AU - Chen Chen
AU - Christian Wachinger
AU - Zhenyu Gong
AU - Cheng Ouyang
AU - Georgios A. Kaissis
AU - Benedikt Wiestler
AU - Daniel Rueckert
PY - 2026
DA - 2026/07/15
JO - Nature Health
DO - 10.1038/s44360-026-00152-8
UR - https://doi.org/10.1038/s44360-026-00152-8
AB - Large language models (LLMs) are increasingly used to answer health-related questions and support healthcare workflows, yet evidence for their safety still relies heavily on static benchmarks that can rapidly become obsolete or be optimized against. Here we introduce a Dynamic, Automatic and Systematic (DAS) red-teaming audit framework that continuously stress-tests LLMs for health across four safety-critical axes: robustness, privacy, bias and hallucination. Validated against board-certified clinicians, a suite of adversarial agents autonomously mutates health-related test cases to uncover vulnerabilities in real time. Applying DAS to 15 state-of-the-art LLMs revealed a profound gap between high static benchmark performance and low dynamic reliability—the ‘benchmarking gap’. Despite median MedQA accuracy exceeding 80%, 94% of previously correct answers failed under dynamic robustness testing. This brittleness generalized to the realistic, open-ended HealthBench dataset, where top-tier models exhibited failure rates exceeding 70%, suggesting that high scores on established static benchmarks may reflect superficial memorization. We observed similarly high failure rates across other domains: privacy leaks were elicited in 86% of scenarios, cognitive bias priming altered recommendations in 81% of fairness tests and hallucination rates exceeded 74% in widely used models. By converting LLM safety evaluation for health from a static checklist into a living adversarial audit, DAS provides a scalable framework for surfacing latent risks before such systems are deployed in consumer-facing health assistants and broader clinical workflows.
SN - 3005-0693
ER -