Utilization of a HIPAA-compliant large language model chatbot in an academic pediatric medical center
7 affiliations
- 1Innovation & Digital Health Accelerator, Boston Children’s Hospital, Boston, Massachusetts, United States of America
- 2Department of Anesthesiology, Critical Care and Pain Medicine, Boston Children’s Hospital, Boston, Massachusetts, United States of America
- 3Department of Anesthesia, Harvard Medical School, Boston, Massachusetts, United States of America
- 4Executive Vice President of Health Affairs, Boston Children’s Hospital, Boston, Massachusetts, United States of America
- 5Chief Information Officer, Boston Children’s Hospital, Boston, Massachusetts, United States of America
- 6Department of Pediatrics, Harvard Medical School, Boston, Massachusetts, United States of America
- 7Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America
Abstract
Despite growing interest in LLM chatbots, little evidence exists on usage patterns in hospital systems. This paper presents a mixed-methods case study of “InternalGPT,” a HIPAA-compliant LLM chatbot made available to employees of the institution. Survey responses and 14 months of system utilization data were analyzed across diverse professional roles in the institution. Of approximately 15,800 hospital employees, 2,149 (13.6%) requested InternalGPT access. Motivations for sign-up aligned with professional roles: operations professionals prioritized administrative automation, clinicians focused on clinical augmentation, and scientists on research and coding. Among InternalGPT users, 52.8% recorded at least one token use; 33.6% never logged-in, and 13.7% logged-in but never used the tool. The top 20% of users consumed 69.4% of tokens. Key barriers to engagement among non-users included limited time for exploration (51.4%) and difficulty using (21.8%), integrating (15.5%), or accessing (9.2%) InternalGPT. Among a small group of employees with sustained InternalGPT use (n = 461), a subset who responded to a self-report survey (n = 92) indicated a mean productivity gain of 30%, corresponding to an exploratory perceived productivity value of $6.3M to $18.9M under varying extrapolation assumptions. Although most hospital employees never used the HIPAA-compliant LLM chatbot, a small minority who utilized it regularly reported substantial productivity improvements, highlighting both the value proposition of LLMs in healthcare and potential for increased engagement. Implementation and knowledge challenges were more frequently cited than AI-feature and performance concerns as key barriers to adoption. These findings underscore that successful LLM implementation in healthcare hinges not only on advanced AI technology, but also on robust system capacity building including investments in user education and technical integration.
Related
- Nutritional status, clinical burden, and healthcare utilization among pediatric outpatients with congenital heart disease: A retrospective cross-sectional study from Indonesia
- The ADHD advantage: Comorbid attention-deficit/hyperactivity disorder is associated with earlier diagnosis and better academic outcomes in children with specific language disorder
- 3D Human Neuromuscular Architecture Encoded by Compliant Patterned Hydrogel Scaffolds
- Design and validation of a GMP-compliant manufacturing platform of extracellular vesicles from mesenchymal stromal cells
- QShield-ZTN: A quantum-resilient, standards-compliant secure orchestration model for autonomous 6G zero-touch networks
BibTeX
@article{rader2026utilization,
title = {{Utilization of a HIPAA-compliant large language model chatbot in an academic pediatric medical center}},
author = {Benjamin Rader and Dinesh K. Rai and Melissa L. Stewart and Sarah Lejfer and Sampath Chennuri and James Hartwell and Victor Moyano and Max H. Witwer and Peter C. Laussen and Heather L. Nelson and John S. Brownstein},
journal = {PLOS Digital Health},
year = {2026},
month = sep,
doi = {10.1371/journal.pdig.0001684},
url = {https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001684},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Utilization of a HIPAA-compliant large language model chatbot in an academic pediatric medical center
AU - Benjamin Rader
AU - Dinesh K. Rai
AU - Melissa L. Stewart
AU - Sarah Lejfer
AU - Sampath Chennuri
AU - James Hartwell
AU - Victor Moyano
AU - Max H. Witwer
AU - Peter C. Laussen
AU - Heather L. Nelson
AU - John S. Brownstein
PY - 2026
DA - 2026/09/17
JO - PLOS Digital Health
DO - 10.1371/journal.pdig.0001684
UR - https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001684
AB - Despite growing interest in LLM chatbots, little evidence exists on usage patterns in hospital systems. This paper presents a mixed-methods case study of “InternalGPT,” a HIPAA-compliant LLM chatbot made available to employees of the institution. Survey responses and 14 months of system utilization data were analyzed across diverse professional roles in the institution. Of approximately 15,800 hospital employees, 2,149 (13.6%) requested InternalGPT access. Motivations for sign-up aligned with professional roles: operations professionals prioritized administrative automation, clinicians focused on clinical augmentation, and scientists on research and coding. Among InternalGPT users, 52.8% recorded at least one token use; 33.6% never logged-in, and 13.7% logged-in but never used the tool. The top 20% of users consumed 69.4% of tokens. Key barriers to engagement among non-users included limited time for exploration (51.4%) and difficulty using (21.8%), integrating (15.5%), or accessing (9.2%) InternalGPT. Among a small group of employees with sustained InternalGPT use (n = 461), a subset who responded to a self-report survey (n = 92) indicated a mean productivity gain of 30%, corresponding to an exploratory perceived productivity value of $6.3M to $18.9M under varying extrapolation assumptions. Although most hospital employees never used the HIPAA-compliant LLM chatbot, a small minority who utilized it regularly reported substantial productivity improvements, highlighting both the value proposition of LLMs in healthcare and potential for increased engagement. Implementation and knowledge challenges were more frequently cited than AI-feature and performance concerns as key barriers to adoption. These findings underscore that successful LLM implementation in healthcare hinges not only on advanced AI technology, but also on robust system capacity building including investments in user education and technical integration.
SN - 2767-3170
ER -