Validating LLM judges for automated oversight of patient communication
- 1Brigham and Women's Hospital
Abstract
LLMs are increasingly used to mediate patient communication, yet scalable evaluation of their safety, accuracy, and communication quality remains an open problem. LLM judges have emerged as automated evaluators, but whether they can holistically replicate human expert judgment is unvalidated. Informed consent for clinical trials presents a demanding case for such validation because it requires conveying complex information to lay audiences under ethical and safety constraints. We developed a stakeholder-informed seven-criterion evaluation rubric spanning safety, reliability, and communication quality. Clinician reference ratings showed strong interrater reliability across all criteria. We validated the rubric on the Informed CONsent Benchmark (ICON-Bench) and benchmarked 19 LLM judges across multiple implementation strategies. LLM judges achieved strong clinician agreement for safety screening and factual verification (Spearman{rho} > 0.80) but weaker agreement for communication quality ({rho} < 0.60). Safety-specialized guard models underperformed general-purpose models. Patient advocates rated communication quality lower than both clinicians and LLM judges. These findings support LLM judges for scalable patient communication oversight while demonstrating the need for recalibration to patient-centered evaluation standards.
Related
- Impact of Automated Perioperative Text Messaging on Patient-Initiated Communication After Total Joint Arthroplasty.
- Balancing Timely Access and Human Oversight in Patient-Facing Artificial Intelligence—Waiting for the Human in the Loop
- Developing and Validating a Patient-Centered Measure of Well-Being in Bipolar Disorder: The WELL-B.
- Theory of Artificial Intelligence Adaptation for Linguistic Concordance: Bridging Gaps in Patient-Centered Communication
- Ethical dilemmas and intervention strategies in doctor-patient-family communication in end-of-life pediatric oncology: a narrative review
BibTeX
@misc{xu2026validating,
title = {{Validating LLM judges for automated oversight of patient communication}},
author = {Z. Xu and J. Zeng and S. Zhou and Z. Zhang and T. Heintz and M. Tonneau and A. Yaghoubi and B. Ye and V. Goddla and L. Lehmann and Y.-H. Chen and D. Kozono and A. Revette and J. Maues and T. Brown and P. Catalano and R. H. Mak and D. Dligach and D. Bitterman},
howpublished = {medRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.16.26363176},
url = {https://www.medrxiv.org/content/10.64898/2026.09.16.26363176v1},
note = {CC BY-NC},
}
RIS
TY - UNPB
TI - Validating LLM judges for automated oversight of patient communication
AU - Z. Xu
AU - J. Zeng
AU - S. Zhou
AU - Z. Zhang
AU - T. Heintz
AU - M. Tonneau
AU - A. Yaghoubi
AU - B. Ye
AU - V. Goddla
AU - L. Lehmann
AU - Y.-H. Chen
AU - D. Kozono
AU - A. Revette
AU - J. Maues
AU - T. Brown
AU - P. Catalano
AU - R. H. Mak
AU - D. Dligach
AU - D. Bitterman
PY - 2026
DA - 2026/09/16
PB - medRxiv
DO - 10.64898/2026.09.16.26363176
UR - https://www.medrxiv.org/content/10.64898/2026.09.16.26363176v1
AB - LLMs are increasingly used to mediate patient communication, yet scalable evaluation of their safety, accuracy, and communication quality remains an open problem. LLM judges have emerged as automated evaluators, but whether they can holistically replicate human expert judgment is unvalidated. Informed consent for clinical trials presents a demanding case for such validation because it requires conveying complex information to lay audiences under ethical and safety constraints. We developed a stakeholder-informed seven-criterion evaluation rubric spanning safety, reliability, and communication quality. Clinician reference ratings showed strong interrater reliability across all criteria. We validated the rubric on the Informed CONsent Benchmark (ICON-Bench) and benchmarked 19 LLM judges across multiple implementation strategies. LLM judges achieved strong clinician agreement for safety screening and factual verification (Spearman{rho} > 0.80) but weaker agreement for communication quality ({rho} < 0.60). Safety-specialized guard models underperformed general-purpose models. Patient advocates rated communication quality lower than both clinicians and LLM judges. These findings support LLM judges for scalable patient communication oversight while demonstrating the need for recalibration to patient-centered evaluation standards.
ER -