When conviction outpaces evidence: diagnostic boundary failures of large language models in adult ADHD self-assessment
5 affiliations
- 1Department of Nursing, West China Second University Hospital, Sichuan University
- 2Key Laboratory of Birth Defects and Related Diseases of Women and Children Sichuan University, Ministry of Education
- 3Department of Neurology, West China Second University Hospital, Sichuan University
- 4Department of Plastic and Aesthetic Surgery, The Second Affiliated Hospital of Guilin Medical University
- 5School of Economics, Zhejiang University of Science and Technology
Abstract
Large language models (LLMs) are increasingly used by people seeking information about possible mental-health conditions, including attention-deficit/hyperactivity disorder (ADHD). This raises an important safety question: can these systems distinguish between stronger clinical evidence and a user’s increasing confidence in a self-diagnosis? We evaluated ChatGPT, Gemini, and Doubao using 24 controlled adult ADHD scenarios representing presentations compatible with ADHD, presentations with insufficient evidence for diagnosis, and presentations with plausible alternative explanations. For each scenario, the models were tested as either clinically relevant information accumulated or the user’s confidence in having ADHD increased without corresponding diagnostic evidence. Testing was repeated independently, producing 1,440 responses. Models were more likely to express unsupported diagnostic certainty when user confidence increased without stronger clinical evidence than when clinically informative evidence accumulated. This pattern was most apparent when the available information was insufficient for ADHD diagnosis or when other explanations remained plausible. In contrast, when clinically relevant information was progressively introduced, model responses generally changed in the direction supported by that information. The three models also differed in their overall diagnostic safety. These findings suggest that evaluating mental-health LLMs requires more than assessing factual accuracy. Safe conversational systems should preserve appropriate diagnostic uncertainty and avoid allowing increasing user confidence alone to substitute for clinical evidence.
Related
- Clinical laboratory diagnostic stewardship as boundary infrastructure: a multi-stakeholder qualitative study of digital public health governance in two Chinese regional referral hospitals
- When fluency outpaces provenance: source traceability and digital health literacy against AI health misinformation
- Diagnostic Assessment of Tamoxifen-Associated Endometrial Pathology: A Narrative Review of Biomarkers, Risk Models, and Translational Gaps
- Diagnostic Accuracy of Pressure Injury Staging Among Nurses: A Multisite Study Using Knowledge Assessment and Image‐Based Evaluation
- Improving Immediate Postpartum Detection of Obstetric Anal Sphincter Injury: Added Diagnostic Value of Impedance Spectroscopy Following Clinical Assessment
BibTeX
@article{qin2026when,
title = {{When conviction outpaces evidence: diagnostic boundary failures of large language models in adult ADHD self-assessment}},
author = {Jiajia Qin and Lu Bai and Chujie Qu and Xiao Wei and Xuchen Luo and Huan Zhou},
journal = {Frontiers in Psychiatry},
year = {2026},
month = nov,
doi = {10.3389/fpsyt.2026.1964950},
url = {https://www.frontiersin.org/articles/10.3389/fpsyt.2026.1964950/full},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - When conviction outpaces evidence: diagnostic boundary failures of large language models in adult ADHD self-assessment
AU - Jiajia Qin
AU - Lu Bai
AU - Chujie Qu
AU - Xiao Wei
AU - Xuchen Luo
AU - Huan Zhou
PY - 2026
DA - 2026/11/06
JO - Frontiers in Psychiatry
DO - 10.3389/fpsyt.2026.1964950
UR - https://www.frontiersin.org/articles/10.3389/fpsyt.2026.1964950/full
AB - Large language models (LLMs) are increasingly used by people seeking information about possible mental-health conditions, including attention-deficit/hyperactivity disorder (ADHD). This raises an important safety question: can these systems distinguish between stronger clinical evidence and a user’s increasing confidence in a self-diagnosis? We evaluated ChatGPT, Gemini, and Doubao using 24 controlled adult ADHD scenarios representing presentations compatible with ADHD, presentations with insufficient evidence for diagnosis, and presentations with plausible alternative explanations. For each scenario, the models were tested as either clinically relevant information accumulated or the user’s confidence in having ADHD increased without corresponding diagnostic evidence. Testing was repeated independently, producing 1,440 responses. Models were more likely to express unsupported diagnostic certainty when user confidence increased without stronger clinical evidence than when clinically informative evidence accumulated. This pattern was most apparent when the available information was insufficient for ADHD diagnosis or when other explanations remained plausible. In contrast, when clinically relevant information was progressively introduced, model responses generally changed in the direction supported by that information. The three models also differed in their overall diagnostic safety. These findings suggest that evaluating mental-health LLMs requires more than assessing factual accuracy. Safe conversational systems should preserve appropriate diagnostic uncertainty and avoid allowing increasing user confidence alone to substitute for clinical evidence.
SN - 1664-0640
ER -