Decision Support in Publicly Available Patient Information Policies at U.S. Osteopathic Medical Schools: A Vignette-Based Document Analysis
- 1College of Osteopathic Medicine, Michigan State University, East Lansing, MI, USA
Abstract
Research ObjectivesTo evaluate whether publicly available institutional guidance supports patient-information decisions across scenarios and schools, and characterize document synthesis and evaluation consistency.
MethodsWe conducted an exploratory, vignette-based document analysis of a geographically diverse nonprobability sample of 20 U.S. osteopathic medical schools. Eight educational vignettes yielded 160 school-vignette pairs. Each pair underwent three separate AI-assisted retrieval-and-evaluation runs, classifying decision support as Explicitly Supported, Inferable, Ambiguous, or Not Addressed. Response selection prioritized greater support for discordant pairs, followed by fewer contributing documents and run order. One investigator verified or revised selected discordant classifications against cited evidence. Explicitly Supported and Inferable were grouped post hoc as sufficient decision support. Analyses were descriptive and included an exploratory two-school model-investigator comparison.
ResultsAt least one eligible source was retrieved for 159 of 160 pairs (99.4%). Final classifications were Explicitly Supported for 18 pairs (11.3%), Inferable for 3 (1.9%), Ambiguous for 132 (82.5%), and Not Addressed for 7 (4.4%). Sufficient support occurred in 21 pairs (13.1%), most frequently for generative AI-assisted reflective writing (7/20 schools, 35%), and in none for personal cloud notes or official clinical logs. Ten schools had no sufficiently supported vignette; the maximum was four of eight. Multiple documents contributed to 111 evaluations (69.4%). Three-run ratings were unanimous for 113 pairs (70.6%), with 80.2% pairwise exact agreement. Investigator review retained 44 of 47 selected discordant ratings and revised three upward. In the two-school comparison, the investigator more frequently judged evidence sufficient when models judged it insufficient than the reverse.
ConclusionsRelevant public guidance was frequently retrieved, but few school-vignette pairs were classified as providing sufficient scenario-specific decision support. These findings highlight a gap between identifying relevant guidance and determining an appropriate course of action within this evaluation framework. They support institutional review of how student-facing materials explain information use, storage, sharing, and approval requirements. Vignette-based review identifies questions requiring clarification. Evaluation with learners and assessment of internal and clinical-site guidance would help determine how these findings translate to students decisions.
Related
- Moraxella species isolated from blood cultures in Europe (MORAXEu): a multicentre study of epidemiology and antimicrobial susceptibility with complementary phylogenomic analysis of publicly available genomes
- Training load, strength training, and perceived recovery in relation to injury occurrence among competitive middle- and long-distance runners: a secondary analysis of publicly available longitudinal training logs
- The Medicine Quality Monitoring Globe: A public health intelligence system for collecting and curating reports of substandard and falsified medical products from publicly available Internet news media
- Reliability, Quality, and Readability of Freely Available Large Language Models for Clinical Decision Support in Perforated Peptic Ulcer: A Blinded, Multidimensional Study
- Perspectives of European Nutrition Professionals on Healthy Eating Policies: An International Comparative Study of Implementation Gaps and Information Credibility
BibTeX
@misc{dai2026decision,
title = {{Decision Support in Publicly Available Patient Information Policies at U.S. Osteopathic Medical Schools: A Vignette-Based Document Analysis}},
author = {N. Dai and K. L. Waarala},
howpublished = {medRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.18.26363437},
url = {https://www.medrxiv.org/content/10.64898/2026.09.18.26363437v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Decision Support in Publicly Available Patient Information Policies at U.S. Osteopathic Medical Schools: A Vignette-Based Document Analysis
AU - N. Dai
AU - K. L. Waarala
PY - 2026
DA - 2026/09/18
PB - medRxiv
DO - 10.64898/2026.09.18.26363437
UR - https://www.medrxiv.org/content/10.64898/2026.09.18.26363437v1
AB - Research ObjectivesTo evaluate whether publicly available institutional guidance supports patient-information decisions across scenarios and schools, and characterize document synthesis and evaluation consistency. MethodsWe conducted an exploratory, vignette-based document analysis of a geographically diverse nonprobability sample of 20 U.S. osteopathic medical schools. Eight educational vignettes yielded 160 school-vignette pairs. Each pair underwent three separate AI-assisted retrieval-and-evaluation runs, classifying decision support as Explicitly Supported, Inferable, Ambiguous, or Not Addressed. Response selection prioritized greater support for discordant pairs, followed by fewer contributing documents and run order. One investigator verified or revised selected discordant classifications against cited evidence. Explicitly Supported and Inferable were grouped post hoc as sufficient decision support. Analyses were descriptive and included an exploratory two-school model-investigator comparison. ResultsAt least one eligible source was retrieved for 159 of 160 pairs (99.4%). Final classifications were Explicitly Supported for 18 pairs (11.3%), Inferable for 3 (1.9%), Ambiguous for 132 (82.5%), and Not Addressed for 7 (4.4%). Sufficient support occurred in 21 pairs (13.1%), most frequently for generative AI-assisted reflective writing (7/20 schools, 35%), and in none for personal cloud notes or official clinical logs. Ten schools had no sufficiently supported vignette; the maximum was four of eight. Multiple documents contributed to 111 evaluations (69.4%). Three-run ratings were unanimous for 113 pairs (70.6%), with 80.2% pairwise exact agreement. Investigator review retained 44 of 47 selected discordant ratings and revised three upward. In the two-school comparison, the investigator more frequently judged evidence sufficient when models judged it insufficient than the reverse. ConclusionsRelevant public guidance was frequently retrieved, but few school-vignette pairs were classified as providing sufficient scenario-specific decision support. These findings highlight a gap between identifying relevant guidance and determining an appropriate course of action within this evaluation framework. They support institutional review of how student-facing materials explain information use, storage, sharing, and approval requirements. Vignette-based review identifies questions requiring clarification. Evaluation with learners and assessment of internal and clinical-site guidance would help determine how these findings translate to students decisions.
ER -