Sex-based diagnostic bias in AI symptom checkers: a structured, exploratory audit of three platforms.
- 1Department of Social and Behavioral Sciences, New York University School of Global Public Health, New York, NY, United States.
- 2Department of Medicine, Division of General Internal Medicine, Tufts University School of Medicine, Boston, MA, United States.
- 3Department of Medicine, Tufts Medical Center/Tufts Medicine, Boston, MA, United States.
Abstract
Artificial intelligence symptom checkers are used by hundreds of millions of people annually as first-line triage tools, and their outputs may influence whether and how quickly a person seeks emergency care. We conducted a structured, exploratory audit of three widely used platforms, Ada Health, Isabel Healthcare, and Docus AI, across four standardized clinical vignettes [acute coronary syndrome (ACS) with chest pressure, ACS without chest pressure, pulmonary embolism, and panic attack], entering each vignette for matched male and female synthetic patient profiles with sex as the sole variable altered. We observed sex-based divergence in outputs from all three platforms, appearing as differences in differential diagnosis rank, differences in assigned probability, and differences in generated explanatory or advisory text. The direction of this divergence was not uniform: some findings were unfavorable to the female profile and others were favorable to it. Isabel Healthcare ranked Heart Attack first for the male profile but third for the female profile on the ACS with chest pressure vignette, and ranked Heart Attack seventh rather than fifth for the female profile on the panic attack vignette; on the ACS without chest pressure vignette, the same platform ranked Heart Attack first for the female profile, ahead of the male profile. Docus AI assigned a lower ACS probability to the female profile on the ACS without chest pressure vignette while its own text described this presentation as common in women, and applied a mirrored justification, common in males, to support a higher probability for the male profile given identical symptoms; the platform's closing advisory text also described immediate medical attention as crucial for the male profile and prompt evaluation as merely important for the female profile. Ada Health's explanatory text for the female profile stated that heart attacks are slightly more common in men than women, presented alongside a demoted cardiac ranking. These findings indicate that AI symptom checkers can produce sex-based divergence in their outputs, including an internally inconsistent, self-contradictory justification on one platform, and support the case for equity auditing of such systems, with attention to both numerical outputs and generated explanatory text, before and after deployment.
Related
- Are Frontier Large Language Models Safer Than Government-Backed Symptom Checkers for Clinical Self-Triage? A Standardised Vignette Evaluation
- Response pathways and clinical workflow integration of digital symptom management platforms for breast cancer: A scoping review
- Menopausal Symptom Burden and Self-Reported Poor Sleep Quality Among Japanese Working Women Aged 40 Years and Older: An Exploratory Cross-Sectional Study
- Diagnostic Yield and Clinical Phenotype in a Lithuanian Familial Hypercholesterolemia Cohort with Heterogeneous Sequencing Platforms
- Diagnostic antibodies at an inflection point: From discovery platforms to design‐oriented engineering
BibTeX
@article{khan2026sexbased,
title = {{Sex-based diagnostic bias in AI symptom checkers: a structured, exploratory audit of three platforms.}},
author = {Samah Khan and Michael K Paasche-Orlow},
journal = {Frontiers in digital health},
year = {2026},
month = oct,
doi = {10.3389/fdgth.2026.1950777},
url = {https://europepmc.org/article/MED/42923008},
note = {CC BY},
}
RIS
TY - JOUR
TI - Sex-based diagnostic bias in AI symptom checkers: a structured, exploratory audit of three platforms.
AU - Samah Khan
AU - Michael K Paasche-Orlow
PY - 2026
DA - 2026/10/08
JO - Frontiers in digital health
DO - 10.3389/fdgth.2026.1950777
UR - https://europepmc.org/article/MED/42923008
AB - Artificial intelligence symptom checkers are used by hundreds of millions of people annually as first-line triage tools, and their outputs may influence whether and how quickly a person seeks emergency care. We conducted a structured, exploratory audit of three widely used platforms, Ada Health, Isabel Healthcare, and Docus AI, across four standardized clinical vignettes [acute coronary syndrome (ACS) with chest pressure, ACS without chest pressure, pulmonary embolism, and panic attack], entering each vignette for matched male and female synthetic patient profiles with sex as the sole variable altered. We observed sex-based divergence in outputs from all three platforms, appearing as differences in differential diagnosis rank, differences in assigned probability, and differences in generated explanatory or advisory text. The direction of this divergence was not uniform: some findings were unfavorable to the female profile and others were favorable to it. Isabel Healthcare ranked Heart Attack first for the male profile but third for the female profile on the ACS with chest pressure vignette, and ranked Heart Attack seventh rather than fifth for the female profile on the panic attack vignette; on the ACS without chest pressure vignette, the same platform ranked Heart Attack first for the female profile, ahead of the male profile. Docus AI assigned a lower ACS probability to the female profile on the ACS without chest pressure vignette while its own text described this presentation as common in women, and applied a mirrored justification, common in males, to support a higher probability for the male profile given identical symptoms; the platform's closing advisory text also described immediate medical attention as crucial for the male profile and prompt evaluation as merely important for the female profile. Ada Health's explanatory text for the female profile stated that heart attacks are slightly more common in men than women, presented alongside a demoted cardiac ranking. These findings indicate that AI symptom checkers can produce sex-based divergence in their outputs, including an internally inconsistent, self-contradictory justification on one platform, and support the case for equity auditing of such systems, with attention to both numerical outputs and generated explanatory text, before and after deployment.
SN - 2673-253X
ER -