Context-aware AI assistance reduces diagnostic error in chest X-ray interpretation
- 1Department of Computer Science, Taibah University
- 2Department of Information Technology, Faculty of Computing and Information Technology, King Abdulaziz University
Abstract
Static AI assistance assumes uniform benefit, but evidence from AI-supported radiology indicates benefit varies by context. Using data from the public Collab-CXR release (104,800 pathology-level observations, or 20,960 reads, from 326 radiologists across 324 cases), we reframed AI display as a human-centered assistance problem, learning and evaluating offline a context-aware assistance policy. The confirmatory average treatment effect on absolute diagnostic error approached null (-0.0014; SE 0.0015; p = 0.329), indicating that average effects obscure heterogeneity. In held-out evaluation, the adaptive policy achieved MAE 0.1034, versus 0.1060 for always showing AI (reduction 0.0026, 2.48%; cluster-bootstrap p = 0.0067) and 0.1088 for never showing AI (reduction 0.0054). The policy was translated into a three-variable decision tree (algorithm prediction, case difficulty, case-level AI accuracy) achieving 93.97% test concordance and a comparable held-out MAE of 0.1033. Simulation-based sensitivity analysis showed the policy contrast remained positive under modest hidden confounding but grew sensitive at higher levels. We conclude that static assistance is suboptimal in radiology and prospective testing of adaptive, trust-aware AI support is merited. For radiologists, administrators, and AI developers, uniform AI deployment is not the best strategy; selective, context-aware use may offer modest benefit, pending prospective confirmation.
Related
- Citizenship reduces welfare chauvinism - evidence across unemployment benefits, social assistance, and sanctions
- Clinical context enhances interpretation of extracorporeal treatment criteria in lithium poisoning
- CT-Referenced Diagnostic Performance and Complementary Error Patterns of Artificial Intelligence Versus a Single Emergency Physician in Chest Radiograph Interpretation: A Retrospective Paired Diagnostic Accuracy Study
- Diagnostic accuracy of respiratory polygraphy interpretation for pediatric obstructive sleep apnea using a built-in algorithm
- Artificial Scientific Intelligence for Measurement-burden-aware Modelling and Interpretation of Multi-site Bone Mineral Density
BibTeX
@article{nour2026contextaware,
title = {{Context-aware AI assistance reduces diagnostic error in chest X-ray interpretation}},
author = {Redhwan Nour and Adeeb Noor},
journal = {Scientific Reports},
year = {2026},
month = aug,
doi = {10.1038/s41598-026-67997-7},
url = {https://doi.org/10.1038/s41598-026-67997-7},
note = {CC BY},
}
RIS
TY - JOUR
TI - Context-aware AI assistance reduces diagnostic error in chest X-ray interpretation
AU - Redhwan Nour
AU - Adeeb Noor
PY - 2026
DA - 2026/08/01
JO - Scientific Reports
DO - 10.1038/s41598-026-67997-7
UR - https://doi.org/10.1038/s41598-026-67997-7
AB - Static AI assistance assumes uniform benefit, but evidence from AI-supported radiology indicates benefit varies by context. Using data from the public Collab-CXR release (104,800 pathology-level observations, or 20,960 reads, from 326 radiologists across 324 cases), we reframed AI display as a human-centered assistance problem, learning and evaluating offline a context-aware assistance policy. The confirmatory average treatment effect on absolute diagnostic error approached null (-0.0014; SE 0.0015; p = 0.329), indicating that average effects obscure heterogeneity. In held-out evaluation, the adaptive policy achieved MAE 0.1034, versus 0.1060 for always showing AI (reduction 0.0026, 2.48%; cluster-bootstrap p = 0.0067) and 0.1088 for never showing AI (reduction 0.0054). The policy was translated into a three-variable decision tree (algorithm prediction, case difficulty, case-level AI accuracy) achieving 93.97% test concordance and a comparable held-out MAE of 0.1033. Simulation-based sensitivity analysis showed the policy contrast remained positive under modest hidden confounding but grew sensitive at higher levels. We conclude that static assistance is suboptimal in radiology and prospective testing of adaptive, trust-aware AI support is merited. For radiologists, administrators, and AI developers, uniform AI deployment is not the best strategy; selective, context-aware use may offer modest benefit, pending prospective confirmation.
SN - 2045-2322
ER -