The economics of accuracy for medical reasoning with large language models
- 1SKA Labs, Atlanta, Georgia, United States of America
- 2Advanced Product Development, Intuitive Surgical, Inc., Atlanta, Georgia, United States of America
Abstract
Deploying large language models (LLMs) in clinical settings is limited by security, reliability, latency, and accessibility concerns that favor smaller, on-device or on-premise models. However, these smaller models may struggle to meet accuracy requirements. While fine-tuning and retrieval-augmented generation (RAG) can improve domain-specific accuracy, these methods require additional labeled data, technical skill, and infrastructure. In contrast, test-time scaling—allocating extra token-budget during inference—offers a training-free alternative to increasing accuracy. However, the trade-offs between these strategies and their interaction with model size remain poorly understood for medical reasoning. To address this gap, we compare three approaches—test-time scaling, fine-tuning, and context grounding—using the Gemma and MedGemma family of LLMs (Gemma-3 1B, Gemma-3 4B, Gemma-3 27B, MedGemma-4B, and MedGemma 27B) and evaluate these systems across common biomedical question-answering (QA) datasets and a set of recently released medical exam questions with the performance of practicing clinicians available for comparison. We test baseline prompts (direct answer, Chain-of-Thought, and self-consistency) while introducing a new prompting method we call “prompt-chaining for continuous reflection” (PCCR) that forces inference time minimum token-generation budgets. We assess accuracy and tokens-generated, allowing us to investigate the accuracy–efficiency trade-offs across prompting, context-grounding, fine-tuning, and model scales. We discover equivalency point configurations where a smaller model’s accuracy falls within one 95% confidence interval of a larger model’s (typically within 1–4 percentage points) reached through increased reasoning budgets, context-grounding, or fine-tuning. Specific effects are apparent and statistically supported: the benefit of medical fine-tuning grew from +4.6 to +15.7 percentage points (non-overlapping 95% CIs) when paired with self-consistency, and enforced extended reasoning raised MedGemma 27B accuracy from 58.1% to 80.1% (p < 10-5) in the absence of context. We also identify an “overthinking” inflection: when high-quality context is available, extended reasoning beyond roughly 128–256 tokens degrades accuracy by 7–13 percentage points. Using these empirical results, we formulate a general framework with equations to balance cost-benefit trade-offs when engineering LLM-based systems for medical reasoning and QA. We recommend generalizable configurations, designs, and patterns to achieve accuracy and efficiency objectives for example use-cases relevant to healthcare organizations. Author summary: When doctors and hospitals want to use artificial intelligence for medical tasks, they face difficult choices. The most capable AI systems are expensive to run and require sending sensitive patient and hospital data to external servers. Smaller systems that can run locally are more practical but may be less accurate. Our research asked: with what configurations can we make smaller AI systems perform as well as larger ones for medical reasoning? We tested five AI models of different sizes, including both generalist and medically-specialized models, on thousands of medical questions using various prompting strategies, including a new and very simple method we developed that encourages the AI to reason more extensively. We discovered several surprising findings which we describe in detail. Most importantly, we found that smaller, cheaper models can match larger ones when given the right combination of prompting strategy, specialized training, and supporting information. We translated these findings into practical guidelines that can help choose AI configurations that balance accuracy, speed, and cost. Our framework could help make medical AI more accessible to institutions with limited computational resources while achieving the highest possible accuracy.
Related
- Experimental reasoning in large language models
- Evidence Scaling for Zero-Shot Protein Reasoning with Large Language Models
- OmicsLM: A Multimodal Large Language Model for Multi-Sample Omics Reasoning
- Enhancing medical education with AI: A comparative study of various large language models in radiology
- Relation-aware dynamic knowledge subgraph enhanced large language models for medical multiple-choice question answering
BibTeX
@article{bhattacharyya2026economics,
title = {{The economics of accuracy for medical reasoning with large language models}},
author = {Kiran Bhattacharyya and Sreeram Kamabattula},
journal = {PLOS Digital Health},
year = {2026},
month = sep,
doi = {10.1371/journal.pdig.0001182},
url = {https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001182},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - The economics of accuracy for medical reasoning with large language models
AU - Kiran Bhattacharyya
AU - Sreeram Kamabattula
PY - 2026
DA - 2026/09/18
JO - PLOS Digital Health
DO - 10.1371/journal.pdig.0001182
UR - https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001182
AB - Deploying large language models (LLMs) in clinical settings is limited by security, reliability, latency, and accessibility concerns that favor smaller, on-device or on-premise models. However, these smaller models may struggle to meet accuracy requirements. While fine-tuning and retrieval-augmented generation (RAG) can improve domain-specific accuracy, these methods require additional labeled data, technical skill, and infrastructure. In contrast, test-time scaling—allocating extra token-budget during inference—offers a training-free alternative to increasing accuracy. However, the trade-offs between these strategies and their interaction with model size remain poorly understood for medical reasoning. To address this gap, we compare three approaches—test-time scaling, fine-tuning, and context grounding—using the Gemma and MedGemma family of LLMs (Gemma-3 1B, Gemma-3 4B, Gemma-3 27B, MedGemma-4B, and MedGemma 27B) and evaluate these systems across common biomedical question-answering (QA) datasets and a set of recently released medical exam questions with the performance of practicing clinicians available for comparison. We test baseline prompts (direct answer, Chain-of-Thought, and self-consistency) while introducing a new prompting method we call “prompt-chaining for continuous reflection” (PCCR) that forces inference time minimum token-generation budgets. We assess accuracy and tokens-generated, allowing us to investigate the accuracy–efficiency trade-offs across prompting, context-grounding, fine-tuning, and model scales. We discover equivalency point configurations where a smaller model’s accuracy falls within one 95% confidence interval of a larger model’s (typically within 1–4 percentage points) reached through increased reasoning budgets, context-grounding, or fine-tuning. Specific effects are apparent and statistically supported: the benefit of medical fine-tuning grew from +4.6 to +15.7 percentage points (non-overlapping 95% CIs) when paired with self-consistency, and enforced extended reasoning raised MedGemma 27B accuracy from 58.1% to 80.1% (p < 10-5) in the absence of context. We also identify an “overthinking” inflection: when high-quality context is available, extended reasoning beyond roughly 128–256 tokens degrades accuracy by 7–13 percentage points. Using these empirical results, we formulate a general framework with equations to balance cost-benefit trade-offs when engineering LLM-based systems for medical reasoning and QA. We recommend generalizable configurations, designs, and patterns to achieve accuracy and efficiency objectives for example use-cases relevant to healthcare organizations. Author summary: When doctors and hospitals want to use artificial intelligence for medical tasks, they face difficult choices. The most capable AI systems are expensive to run and require sending sensitive patient and hospital data to external servers. Smaller systems that can run locally are more practical but may be less accurate. Our research asked: with what configurations can we make smaller AI systems perform as well as larger ones for medical reasoning? We tested five AI models of different sizes, including both generalist and medically-specialized models, on thousands of medical questions using various prompting strategies, including a new and very simple method we developed that encourages the AI to reason more extensively. We discovered several surprising findings which we describe in detail. Most importantly, we found that smaller, cheaper models can match larger ones when given the right combination of prompting strategy, specialized training, and supporting information. We translated these findings into practical guidelines that can help choose AI configurations that balance accuracy, speed, and cost. Our framework could help make medical AI more accessible to institutions with limited computational resources while achieving the highest possible accuracy.
SN - 2767-3170
ER -