M3: Conversational LLMs simplify secure clinical data access, understanding, and analysis
6 affiliations
- 1Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America
- 2Department of Computer Science, Technical University of Munich, Munich, Germany
- 3Department of Engineering, Universitat Pompeu Fabra, Barcelona, Spain
- 4Office of Data Science, St. Jude Children’s Research Hospital, Memphis, Tennessee, United States of America
- 5Beth Israel Deaconess Medical Center, Boston, Massachusetts, United States of America
- 6Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, United States of America
Abstract
Large-scale clinical databases offer opportunities for medical research, but their complexity creates barriers to effective use. The Medical Information Mart for Intensive Care (MIMIC-IV), one of the world’s largest open-source electronic health record databases, traditionally requires both SQL proficiency and clinical domain expertise. We introduce M3, a system that enables natural language querying of MIMIC-IV data through the Model Context Protocol. With a single command, M3 retrieves MIMIC-IV from PhysioNet, launches a local SQLite instance or connects to hosted BigQuery, and allows researchers to pose clinical questions in plain English. We evaluated M3 using samples from the EHRSQL 2024 benchmark with two language models. On one hundred answerable questions, the proprietary Claude Sonnet 4 achieved 94% accuracy and the open-weights gpt-oss-20B (deployable locally on consumer hardware) achieved 93%; on a matched sample of one hundred unanswerable questions, where correct behavior is to abstain rather than produce SQL, gpt-oss-20B correctly abstained on 69%. Both models translate natural language into SQL, execute queries against MIMIC-IV, and return structured results alongside the underlying query for verification. Error analysis revealed that most failures stemmed from complex temporal reasoning or ambiguous question phrasing rather than fundamental architectural limitations. The comparable performance of a smaller open-weights model demonstrates that privacy-preserving local deployment is viable for sensitive clinical data analysis. M3 lowers technical barriers to critical care data analysis and is designed with security measures including OAuth2 authentication, query validation, and audit logging. Author summary: Clinical databases like MIMIC-IV contain detailed patient records that could advance medical knowledge, but analyzing them requires specialized programming skills and understanding of clinical workflows. We developed M3, a tool that lets researchers ask questions about MIMIC-IV data in everyday language instead of writing complex database queries. We tested M3 with two different artificial intelligence models: a commercial model (Claude Sonnet 4) and an open-weights model (gpt-oss-20B) that can run locally on a laptop. Both models answered 93–94% of clinical questions correctly by automatically generating and executing appropriate database queries. When errors occurred, they usually involved questions with multiple valid interpretations or complex time-based logic. The similar performance of the smaller, open-weights model means that researchers concerned about data privacy can analyze clinical information on their own computers without sending queries to external servers. M3 makes clinical data analysis accessible to more researchers while maintaining security protections required for medical information.
Related
- OSIRIS-MCP: Towards secure conversational access to research information in OSIRIS
- Discovery of a hydrolysis operon in Lachnospiraceae provides a genetic basis to understanding castalagin metabolism by the human microbiome in a clinical study.
- Understanding the processes involved in sepsis recognition across clinical contexts to guide individualised management: A phenomenographic study
- Understanding when and how knowledge graphs add value to EHR-based clinical prediction across Parkinson's disease, multiple sclerosis, and chronic pain
- Understanding Public Trust in AI-Based Clinical Psychological Decision-Making: A Qualitative Study
BibTeX
@article{attrach2026conversational,
title = {{M3: Conversational LLMs simplify secure clinical data access, understanding, and analysis}},
author = {Rafi Al Attrach and Pedro Moreira and Rajna Fani and Renato Umeton and Amelia Fiske and Leo Anthony Celi},
journal = {PLOS Digital Health},
year = {2026},
month = sep,
doi = {10.1371/journal.pdig.0001671},
url = {https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001671},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - M3: Conversational LLMs simplify secure clinical data access, understanding, and analysis
AU - Rafi Al Attrach
AU - Pedro Moreira
AU - Rajna Fani
AU - Renato Umeton
AU - Amelia Fiske
AU - Leo Anthony Celi
PY - 2026
DA - 2026/09/17
JO - PLOS Digital Health
DO - 10.1371/journal.pdig.0001671
UR - https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001671
AB - Large-scale clinical databases offer opportunities for medical research, but their complexity creates barriers to effective use. The Medical Information Mart for Intensive Care (MIMIC-IV), one of the world’s largest open-source electronic health record databases, traditionally requires both SQL proficiency and clinical domain expertise. We introduce M3, a system that enables natural language querying of MIMIC-IV data through the Model Context Protocol. With a single command, M3 retrieves MIMIC-IV from PhysioNet, launches a local SQLite instance or connects to hosted BigQuery, and allows researchers to pose clinical questions in plain English. We evaluated M3 using samples from the EHRSQL 2024 benchmark with two language models. On one hundred answerable questions, the proprietary Claude Sonnet 4 achieved 94% accuracy and the open-weights gpt-oss-20B (deployable locally on consumer hardware) achieved 93%; on a matched sample of one hundred unanswerable questions, where correct behavior is to abstain rather than produce SQL, gpt-oss-20B correctly abstained on 69%. Both models translate natural language into SQL, execute queries against MIMIC-IV, and return structured results alongside the underlying query for verification. Error analysis revealed that most failures stemmed from complex temporal reasoning or ambiguous question phrasing rather than fundamental architectural limitations. The comparable performance of a smaller open-weights model demonstrates that privacy-preserving local deployment is viable for sensitive clinical data analysis. M3 lowers technical barriers to critical care data analysis and is designed with security measures including OAuth2 authentication, query validation, and audit logging. Author summary: Clinical databases like MIMIC-IV contain detailed patient records that could advance medical knowledge, but analyzing them requires specialized programming skills and understanding of clinical workflows. We developed M3, a tool that lets researchers ask questions about MIMIC-IV data in everyday language instead of writing complex database queries. We tested M3 with two different artificial intelligence models: a commercial model (Claude Sonnet 4) and an open-weights model (gpt-oss-20B) that can run locally on a laptop. Both models answered 93–94% of clinical questions correctly by automatically generating and executing appropriate database queries. When errors occurred, they usually involved questions with multiple valid interpretations or complex time-based logic. The similar performance of the smaller, open-weights model means that researchers concerned about data privacy can analyze clinical information on their own computers without sending queries to external servers. M3 makes clinical data analysis accessible to more researchers while maintaining security protections required for medical information.
SN - 2767-3170
ER -