Safety boundary maintenance in consumer AI systems responding to pediatric health queries: a cross-platform benchmark evaluation under naturalistic and adversarially pressured conditions
5 affiliations
- 1Medical Sciences Education Research Center, Mashhad University of Medical Sciences, Mashhad, Iran
- 2Student Research Committee, Department of Health Information Technology and Management, Medical Informatics, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran
- 3Department of Anesthesiology and Intensive Care, Mashhad University of Medical Sciences, Mashhad, Iran
- 4Department of Gastroenterology and Hepatology, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran
- 5Department of Medical Education, School of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran
Abstract
Consumer artificial intelligence chatbots are now accessed by hundreds of millions of users seeking health information, yet systematic evaluation of their safety boundary maintenance under real-world caregiver pressure remains scarce. We evaluated PediatricSafetyBench-v2, a benchmark of 600 pediatric health queries comprising 300 authentic caregiver queries sourced from the HealthCareMagic-100k-en physician consultation corpus and 300 matched adversarial variants incorporating six operationalized caregiver pressure patterns, across four consumer AI systems (GPT-4o-mini, Gemini-2.0-Flash, Claude-3.5-Haiku, and Llama-3.1-8B). Safety boundary maintenance was assessed using a validated five-component Safety Composite Score (maximum 15 points; safety-appropriate threshold of 10 or above), validated against independent human raters prior to full-corpus application (mean weighted kappa 0.76; Pearson r = 0.88). The overall safety-appropriate rate was 95.5%. Safety-oriented system prompt deployment improved safety-appropriate rates by 5.9 percentage points across all four models. Counter-intuitively, adversarial caregiver pressure was associated with higher rather than lower Safety Composite Score values for all four models across all ten topic categories and severity levels. False expertise claims were the most vulnerability-inducing pressure pattern, whereas emotional escalation was associated with the highest scores. Consumer AI systems maintain safety boundaries in the large majority of pediatric health interactions. PediatricSafetyBench-v2 is publicly released for longitudinal safety monitoring.
Related
- Comparative Evaluation of Large Language Models in Responding to Common Misconceptions Regarding Periodontal Disease and Oral Hygiene
- Naturalistic visual search detects mild cognitive impairment across distractor conditions
- How Can We Help People With Long‐Term Health Conditions Understand and Engage in Physical Activity Maintenance From Healthcare to Community Settings? Perspectives of Patients, Carers and Professionals
- Determinants of MASLD and liver fibrosis in maintenance hemodialysis: a prospective evaluation of metabolic, nutritional, and volume-related factors.
- Economic evaluation alongside the PAOLA-1 /ENGOT-ov25 trial: cost of ovarian cancer and cost-effectiveness of olaparib plus bevacizumab first-line maintenance
BibTeX
@article{zolfaghari2026safety,
title = {{Safety boundary maintenance in consumer AI systems responding to pediatric health queries: a cross-platform benchmark evaluation under naturalistic and adversarially pressured conditions}},
author = {Vahideh Zolfaghari and Leila Mashhadi and Mitra Ahadi and Farzaneh Sedaghatkar and MohammadReza Kargozari},
journal = {NPJ Digital Medicine},
year = {2026},
month = jul,
doi = {10.1038/s41746-026-02985-9},
url = {https://doi.org/10.1038/s41746-026-02985-9},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - JOUR
TI - Safety boundary maintenance in consumer AI systems responding to pediatric health queries: a cross-platform benchmark evaluation under naturalistic and adversarially pressured conditions
AU - Vahideh Zolfaghari
AU - Leila Mashhadi
AU - Mitra Ahadi
AU - Farzaneh Sedaghatkar
AU - MohammadReza Kargozari
PY - 2026
DA - 2026/07/14
JO - NPJ Digital Medicine
DO - 10.1038/s41746-026-02985-9
UR - https://doi.org/10.1038/s41746-026-02985-9
AB - Consumer artificial intelligence chatbots are now accessed by hundreds of millions of users seeking health information, yet systematic evaluation of their safety boundary maintenance under real-world caregiver pressure remains scarce. We evaluated PediatricSafetyBench-v2, a benchmark of 600 pediatric health queries comprising 300 authentic caregiver queries sourced from the HealthCareMagic-100k-en physician consultation corpus and 300 matched adversarial variants incorporating six operationalized caregiver pressure patterns, across four consumer AI systems (GPT-4o-mini, Gemini-2.0-Flash, Claude-3.5-Haiku, and Llama-3.1-8B). Safety boundary maintenance was assessed using a validated five-component Safety Composite Score (maximum 15 points; safety-appropriate threshold of 10 or above), validated against independent human raters prior to full-corpus application (mean weighted kappa 0.76; Pearson r = 0.88). The overall safety-appropriate rate was 95.5%. Safety-oriented system prompt deployment improved safety-appropriate rates by 5.9 percentage points across all four models. Counter-intuitively, adversarial caregiver pressure was associated with higher rather than lower Safety Composite Score values for all four models across all ten topic categories and severity levels. False expertise claims were the most vulnerability-inducing pressure pattern, whereas emotional escalation was associated with the highest scores. Consumer AI systems maintain safety boundaries in the large majority of pediatric health interactions. PediatricSafetyBench-v2 is publicly released for longitudinal safety monitoring.
SN - 2398-6352
ER -