Decision-only medical AI: early evidence and pairing with language models
- 1BRIDGE GenAI Lab, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA; The Windreich Department of Artificial Intelligence and Human H
Abstract
Recently introduced decision-only AI returns probabilities over specified options rather than text. We tested the clinical consistency and potential bias susceptibility of the two available decision models, Jev and OpenAI Decisions, against a large language model (LLM), GPT-6 Luna, then paired each decision model with the LLM. Across 2,134 case presentations and 2,483,222 model calls, we measured decision changes after adding labels or pressure cues without changing clinical findings. In the original controlled challenges, these additions changed 13.4 more decisions per 100 than neutral wording for the decision models and 14.0 for the LLM. Pairing reduced this to 2.6, about 81% lower in development and 42% to 48% lower in held-out reports. Triage accuracy stayed near 81% to 82%. The paired Decisions workflow cost at most US$0.77 per 1,000 entries. We see this as a practical proof of concept for reducing framing sensitivity and flagging influential text for review.
Related
- Medical Decision‐Making Capacity in Early‐Stage Alzheimer Patients: Comparison With Healthy Controls and Benefit of Assistive Information
- Real-world evaluation of a multilingual agentic retrieval-augmented clinical AI framework for evidence-grounded medical decision support
- Benchmarking large language models for HIV medical decision support
- MES: A Multi-Agent Evidence Synthesis System for Medical Decision-Making
- Human–artificial intelligence collaboration in emergency medical services: from diagnostic algorithms to shared decision support
BibTeX
@misc{omar2026decisiononly,
title = {{Decision-only medical AI: early evidence and pairing with language models}},
author = {M. Omar and A. Ramaswamy and A. Sakhuja and B. S. Glicksberg and A. Gorenshtein and J. B. Kruskal and M. Ahmed and O. R. Brook and R.-E. E. Abdulnour and A. Tyagi and D. Patel and A. W. Charney and Y. Barash and E. Klang and G. N. Nadkarni},
howpublished = {medRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.21.26366149},
url = {https://www.medrxiv.org/content/10.64898/2026.10.21.26366149v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Decision-only medical AI: early evidence and pairing with language models
AU - M. Omar
AU - A. Ramaswamy
AU - A. Sakhuja
AU - B. S. Glicksberg
AU - A. Gorenshtein
AU - J. B. Kruskal
AU - M. Ahmed
AU - O. R. Brook
AU - R.-E. E. Abdulnour
AU - A. Tyagi
AU - D. Patel
AU - A. W. Charney
AU - Y. Barash
AU - E. Klang
AU - G. N. Nadkarni
PY - 2026
DA - 2026/10/21
PB - medRxiv
DO - 10.64898/2026.10.21.26366149
UR - https://www.medrxiv.org/content/10.64898/2026.10.21.26366149v1
AB - Recently introduced decision-only AI returns probabilities over specified options rather than text. We tested the clinical consistency and potential bias susceptibility of the two available decision models, Jev and OpenAI Decisions, against a large language model (LLM), GPT-6 Luna, then paired each decision model with the LLM. Across 2,134 case presentations and 2,483,222 model calls, we measured decision changes after adding labels or pressure cues without changing clinical findings. In the original controlled challenges, these additions changed 13.4 more decisions per 100 than neutral wording for the decision models and 14.0 for the LLM. Pairing reduced this to 2.6, about 81% lower in development and 42% to 48% lower in held-out reports. Triage accuracy stayed near 81% to 82%. The paired Decisions workflow cost at most US$0.77 per 1,000 entries. We see this as a practical proof of concept for reducing framing sensitivity and flagging influential text for review.
ER -