Physician Agreement and Confidence in Rural Alaska Medevac Decisions: A Cross-Sectional Vignette Study
- 1University of Washington
Abstract
PurposeRural Alaska relies on air ambulances ("medevacs") to connect remote villages off the road system to higher-level care. Yet medevac decision-making is complicated by limited guidelines, high costs, and delayed transport. We aimed to quantify variability in physician medevac decisions and identify contributing factors.
MethodsWe conducted a cross-sectional survey of physicians (n=20) within the Alaska Tribal Health System using 20 clinical vignettes representing varying levels of transport complexity. Participants selected transport modality (medevac, commercial, or remain in village) and rated confidence. Interrater agreement was assessed using Gwets AC1. A generalized linear mixed model evaluated predictors of medevac use.
FindingsAgreement was "almost perfect" for clearly critical cases ("always medevac," AC1=0.97) but declined substantially for intermediate scenarios ("possible medevac" AC1=0.39; "never medevac" AC1=0.26; "special considerations" AC1=0.12). Confidence strongly correlated with agreement (r=0.83, p<0.001), forming distinct clusters of high-, medium-, and low-certainty decisions. Vignette type was the dominant driver of medevac use (OR 455.5 for "always medevac"), but decisions did not change with experience (OR=0.99, 0.94-1.04). Between-physician variability persisted after adjustment (ICC 3.8%; MOR 1.67), consistent with differences in risk tolerance.
ConclusionsMedevac decision-making is highly consistent in certain clinical cases but interrater agreement is limited for intermediate cases. Without a shared gold standard of "correct" medevac decisions, future clinical decision support cannot rely on automated medevac decision making. Efforts to improve care should focus on deconstructing decisions, enhancing outcome feedback, and developing targeted decision support rather than prescriptive decision models.
Related
- Knowledge, Confidence and Preparedness for Brain Death Determination Among Anaesthesiology and Reanimation Residents: A National Web-Based Survey with an Embedded Randomised Vignette.
- AI-Assisted Clinical Decision Support Versus Standard Teleconsultation in Adult Medicine: A Systematic Review of Diagnostic Accuracy, Referral and Testing Decisions, and Clinician Confidence.
- Harmful dimensions of medical culture across physician subgroups: a cross-sectional study of 1002 physicians predominantly practising in Belgium.
- Which Features of Medication Double-Check Procedures are Considered Desirable and Implemented in Practice? A Cross-Sectional Web-Based Vignette Study among Nurses in Japan
- Assessment of diagnostic reasoning in acute vertigo using vignette-based tools: A cross-sectional comparison between general practitioners and final-year medical students
BibTeX
@misc{williams2026physician,
title = {{Physician Agreement and Confidence in Rural Alaska Medevac Decisions: A Cross-Sectional Vignette Study}},
author = {C. Williams and S. Weter and J. Wood and H. R. Saul and B. R. Rice},
howpublished = {medRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.21.26363599},
url = {https://www.medrxiv.org/content/10.64898/2026.09.21.26363599v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Physician Agreement and Confidence in Rural Alaska Medevac Decisions: A Cross-Sectional Vignette Study
AU - C. Williams
AU - S. Weter
AU - J. Wood
AU - H. R. Saul
AU - B. R. Rice
PY - 2026
DA - 2026/09/21
PB - medRxiv
DO - 10.64898/2026.09.21.26363599
UR - https://www.medrxiv.org/content/10.64898/2026.09.21.26363599v1
AB - PurposeRural Alaska relies on air ambulances ("medevacs") to connect remote villages off the road system to higher-level care. Yet medevac decision-making is complicated by limited guidelines, high costs, and delayed transport. We aimed to quantify variability in physician medevac decisions and identify contributing factors. MethodsWe conducted a cross-sectional survey of physicians (n=20) within the Alaska Tribal Health System using 20 clinical vignettes representing varying levels of transport complexity. Participants selected transport modality (medevac, commercial, or remain in village) and rated confidence. Interrater agreement was assessed using Gwets AC1. A generalized linear mixed model evaluated predictors of medevac use. FindingsAgreement was "almost perfect" for clearly critical cases ("always medevac," AC1=0.97) but declined substantially for intermediate scenarios ("possible medevac" AC1=0.39; "never medevac" AC1=0.26; "special considerations" AC1=0.12). Confidence strongly correlated with agreement (r=0.83, p<0.001), forming distinct clusters of high-, medium-, and low-certainty decisions. Vignette type was the dominant driver of medevac use (OR 455.5 for "always medevac"), but decisions did not change with experience (OR=0.99, 0.94-1.04). Between-physician variability persisted after adjustment (ICC 3.8%; MOR 1.67), consistent with differences in risk tolerance. ConclusionsMedevac decision-making is highly consistent in certain clinical cases but interrater agreement is limited for intermediate cases. Without a shared gold standard of "correct" medevac decisions, future clinical decision support cannot rely on automated medevac decision making. Efforts to improve care should focus on deconstructing decisions, enhancing outcome feedback, and developing targeted decision support rather than prescriptive decision models.
ER -