The Online Health Safety Gap: Consensus Alignment Does Not Imply Safety in Peer-to-Peer Health Narratives.
- 1University of Maryland, Baltimore County (UMBC)
Abstract
Online health information systems evaluate content by its alignment with medical consensus, treating that alignment as a reliable signal of safety. In peer-to-peer health discourse, that assumption fails: advice that is factually accurate is not necessarily safe to act on. This paper identifies and empirically validates the Online Health Safety Gap, a structural misspecification in which medically accurate content can carry substantial health risk. Across 704 health narratives (699 classified), two scores computed from non-overlapping feature sets, Narrative Truth Distance for epistemic divergence and Narrative Risk Score for health risk potential, share only 4.9% of their variance (r = 0.222, p < 0.001). Consequently, 39.6% of narratives fall in the two off-diagonal quadrants that single-axis systems mishandle by construction: aligned-but- risky content (25.2%) invisible to fact-checking, and divergent-but-safe content (14.4%) incorrectly suppressed. The same independence constrains what existing benchmarks can measure. On an expert-labeled misinformation benchmark of 437 posts containing 127 misinformation instances, the highest discrimination attained is Youdens J of 0.349, by a supervised classifier trained on those labels directly, which neither frozen biomedical embeddings nor a prompted large language model improves upon. Labels encoding factual accuracy therefore carry little signal about behavioral risk, and benchmarks of this construction cannot evaluate risk-aware assessment. To operationalize the gap, we introduce the Classification Quadrant, which maps epistemic divergence and health risk potential onto four governance categories with distinct intervention implications, and argue for a shift from fact- centric evaluation to risk-aware assessment.
Related
- Developing a consensus patient safety curriculum for junior doctors in Taiwan: a modified Delphi protocol
- Collecting health narratives at scale: A multi-study evaluation of speech-to-text in health surveys
- RadPulse: A Free, National, Near-Peer Online Prescribing Safety Assessment Preparation Course for United Kingdom Final-Year Medical Students
- Trust in HPV vaccine effectiveness and safety and willingness to initiate HPV vaccination among unvaccinated female college students recruited through university-based online networks in China: a cross-sectional study
- Performance of five large language models in frozen shoulder patient education: a multidimensional evaluation of readability, quality, clinical alignment, and safety
BibTeX
@misc{clark2026online,
title = {{The Online Health Safety Gap: Consensus Alignment Does Not Imply Safety in Peer-to-Peer Health Narratives.}},
author = {O. Clark and K. Joshi and Z. Ahmed},
howpublished = {medRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.18.26363448},
url = {https://www.medrxiv.org/content/10.64898/2026.09.18.26363448v1},
note = {CC BY-ND},
}
RIS
TY - UNPB
TI - The Online Health Safety Gap: Consensus Alignment Does Not Imply Safety in Peer-to-Peer Health Narratives.
AU - O. Clark
AU - K. Joshi
AU - Z. Ahmed
PY - 2026
DA - 2026/09/18
PB - medRxiv
DO - 10.64898/2026.09.18.26363448
UR - https://www.medrxiv.org/content/10.64898/2026.09.18.26363448v1
AB - Online health information systems evaluate content by its alignment with medical consensus, treating that alignment as a reliable signal of safety. In peer-to-peer health discourse, that assumption fails: advice that is factually accurate is not necessarily safe to act on. This paper identifies and empirically validates the Online Health Safety Gap, a structural misspecification in which medically accurate content can carry substantial health risk. Across 704 health narratives (699 classified), two scores computed from non-overlapping feature sets, Narrative Truth Distance for epistemic divergence and Narrative Risk Score for health risk potential, share only 4.9% of their variance (r = 0.222, p < 0.001). Consequently, 39.6% of narratives fall in the two off-diagonal quadrants that single-axis systems mishandle by construction: aligned-but- risky content (25.2%) invisible to fact-checking, and divergent-but-safe content (14.4%) incorrectly suppressed. The same independence constrains what existing benchmarks can measure. On an expert-labeled misinformation benchmark of 437 posts containing 127 misinformation instances, the highest discrimination attained is Youdens J of 0.349, by a supervised classifier trained on those labels directly, which neither frozen biomedical embeddings nor a prompted large language model improves upon. Labels encoding factual accuracy therefore carry little signal about behavioral risk, and benchmarks of this construction cannot evaluate risk-aware assessment. To operationalize the gap, we introduce the Classification Quadrant, which maps epistemic divergence and health risk potential onto four governance categories with distinct intervention implications, and argue for a shift from fact- centric evaluation to risk-aware assessment.
ER -