Teaching LLMs How ICU Physicians Approach Clinical Reasoning Through OMOP-Aligned Retrieval Improves Reasoning Across Clinical Domains
Abstract
Clinical decision-making relies on identifying relevant patient information to guide diagnosis and treatment, a challenge that is especially difficult in the data-dense and rapidly changing intensive care unit (ICU). Large language models (LLMs) could support this task. However, existing applications and datasets mostly emphasize surface-level retrieval or factual recall rather than the inductive and deductive reasoning clinicians practice to select and reason over decision-relevant evidence. We hypothesized that training LLMs on expert ICU reasoning could yield clinical reasoning skills that generalize beyond critical care. Here we introduce ICU-REACT, a reasoning dataset developed with 19 clinicians through a clinician-in-the-loop framework to teach LLMs to perform information retrieval and context-aware clinical reasoning in the ICU. Using ICU-REACT, we fine-tuned Clin-REACT models spanning 8B-70B parameters and three model families. Across five clinical reasoning benchmarks, Clin-REACT consistently outperformed its backbone models and open-source general-purpose and medical LLMs. Gains extended to different tasks including script concordance tests, and downstream diagnosis and treatment tasks. These findings suggest that expert reasoning supervision in critical care can improve broader clinical reasoning, although prospective evaluation is needed before real-world clinical use.
Related
- Effectiveness of Generative AI–Enabled Teaching for Clinical Reasoning in Health Care Education: Protocol for a Systematic Review and Meta-Analysis
- Temporal-spatial reasoning over hypergraph knowledge structures for multimodal retrieval-augmented generation
- Preserving Clinical Reasoning in ChatGPT-Integrated Medical Education: From AI Use to Cognitive Engagement.
- Influence of large language models on pediatricians’ clinical reasoning and decision-making process
- Enhancing Evidence Translation Through Structured Clinical Reasoning: A Mixed-Methods Controlled Feasibility Study of Peer Visibility and Readiness Mechanisms in Applied Learning
BibTeX
@misc{contreras2026teaching,
title = {{Teaching LLMs How ICU Physicians Approach Clinical Reasoning Through OMOP-Aligned Retrieval Improves Reasoning Across Clinical Domains}},
author = {Miguel Contreras and Scott Siegel and Subhash Nerella and Jessica Sena and Jiaqing Zhang and Heng Sun and Hruday Tej Akkaladevi and Peiyu Lu and Jordan Rosen and Sumit Kapoor and Sasank Desaraju and Grace R. Thompson and Jacob Purcell and Michael Petrauskis and Philip KW. Hong and Meghan Brennan and Sarah Chrabaszcz and Tierra Smith and Ronnie Ren and Michel S. Kabbash and Ceyhun Haziroglu and Rushi Patel and Gabriel Gomez and Charlotte Chaiklin and Randy Leung and Kenneth N. John and Whitman Wiggins and Philip Kayser and Vincent Bird and Maria Bruzzone and Tyler J. Loftus and Azra Bihorac and Parisa Rashidi},
howpublished = {Research Square},
year = {2026},
month = oct,
doi = {10.21203/rs.3.rs-11157053/v1},
url = {https://doi.org/10.21203/rs.3.rs-11157053/v1},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Teaching LLMs How ICU Physicians Approach Clinical Reasoning Through OMOP-Aligned Retrieval Improves Reasoning Across Clinical Domains
AU - Miguel Contreras
AU - Scott Siegel
AU - Subhash Nerella
AU - Jessica Sena
AU - Jiaqing Zhang
AU - Heng Sun
AU - Hruday Tej Akkaladevi
AU - Peiyu Lu
AU - Jordan Rosen
AU - Sumit Kapoor
AU - Sasank Desaraju
AU - Grace R. Thompson
AU - Jacob Purcell
AU - Michael Petrauskis
AU - Philip KW. Hong
AU - Meghan Brennan
AU - Sarah Chrabaszcz
AU - Tierra Smith
AU - Ronnie Ren
AU - Michel S. Kabbash
AU - Ceyhun Haziroglu
AU - Rushi Patel
AU - Gabriel Gomez
AU - Charlotte Chaiklin
AU - Randy Leung
AU - Kenneth N. John
AU - Whitman Wiggins
AU - Philip Kayser
AU - Vincent Bird
AU - Maria Bruzzone
AU - Tyler J. Loftus
AU - Azra Bihorac
AU - Parisa Rashidi
PY - 2026
DA - 2026/10/14
PB - Research Square
DO - 10.21203/rs.3.rs-11157053/v1
UR - https://doi.org/10.21203/rs.3.rs-11157053/v1
AB - Clinical decision-making relies on identifying relevant patient information to guide diagnosis and treatment, a challenge that is especially difficult in the data-dense and rapidly changing intensive care unit (ICU). Large language models (LLMs) could support this task. However, existing applications and datasets mostly emphasize surface-level retrieval or factual recall rather than the inductive and deductive reasoning clinicians practice to select and reason over decision-relevant evidence. We hypothesized that training LLMs on expert ICU reasoning could yield clinical reasoning skills that generalize beyond critical care. Here we introduce ICU-REACT, a reasoning dataset developed with 19 clinicians through a clinician-in-the-loop framework to teach LLMs to perform information retrieval and context-aware clinical reasoning in the ICU. Using ICU-REACT, we fine-tuned Clin-REACT models spanning 8B-70B parameters and three model families. Across five clinical reasoning benchmarks, Clin-REACT consistently outperformed its backbone models and open-source general-purpose and medical LLMs. Gains extended to different tasks including script concordance tests, and downstream diagnosis and treatment tasks. These findings suggest that expert reasoning supervision in critical care can improve broader clinical reasoning, although prospective evaluation is needed before real-world clinical use.
SN - 2693-5015
ER -