Skip to content
DraconiticResearch
Search Sources API
203,952 studies31 sourcesUpdated just now
PreprintCC BY 4.0Zenodo

WardBench: Evaluating Policy Compliance, Dynamic FHIR Consent, and Construct Validity in Clinical EHR Agents

  • Vadim Nareyko
Open at sourcePDFDOICite

Abstract

WardBench is a synthetic FHIR R4 agent benchmark for joint utility and policy compliance under role, HL7 PurposeOfUse, patient-stateful FHIR Consent, and PSY labels. Built on Synthea (seed 20261019): 120 main tasks (allowed, denied, boundary, adversarial), plus control and permitted twins, graded deterministically on trajectories (access, action, disclosure).

This draft reports a local 11-model evaluation (S1-S20b).

Draft status: held-out official evaluation is pending clinician/DPO review of the assumed policy table, results are not a legal compliance certification (HIPAA/GDPR/EHDS).

Code and review packet: https://github.com/WardBench/wardbench

Related

  1. How reliable are clinical balance and ataxia scales in healthy children aged 4-12 years? An inter-rater reliability and construct validity study of the PBS, SARA, and BARS.
    Elif Acar Arslan, Ece Gültekin, Gülsena Utku Umut +4
    Frontiers in neurologyPeer-reviewedCC BYFull textNeurologyEurope PMCCrossref
  2. Revisiting the Construct Validity of the DASS-21 in Thai Young People: Evidence for a General Distress Factor from a Cross-Sectional Psychometric Study
    Nopporn Vongsirimas, Phunagphet Kesornsamut, Wareerat Thanoi +5
    Nursing reports (Pavia, Italy)Peer-reviewedCC BYFull textEurope PMC
  3. Construct validity, responsiveness, and minimal important change of the 5 times sit-to-stand test when used with healthy community-living older adults
    Daniel C. Low, Aliah F. Shaheen, Liam C. O’Reilly +5
    Scientific ReportsPeer-reviewedCC BY-NC-ND 4.0Full textCrossref
  4. Assessing the construct validity of cross-national harmonized cognitive assessments using age.
    Sneha Sarah Mani, M Maria Glymour, Eleanor Hayes-Larson +8
    NPJ dementiaPeer-reviewedCC BYFull textPsychiatry & mental healthEurope PMC
  5. Reproducibility is not construct validity: LLM measurement of institutionally situated communication
    Veronika Batzdorfer, Carlo Romano Marcello Alessandro Santagiustina
    HALPreprintCC BY-NC-ND 4.0Full textStatisticsHAL

Cite

BibTeXRIS
BibTeX
@misc{nareyko2026wardbench,
  title = {{WardBench: Evaluating Policy Compliance, Dynamic FHIR Consent, and Construct Validity in Clinical EHR Agents}},
  author = {Vadim Nareyko},
  howpublished = {Zenodo},
  year = {2026},
  month = oct,
  doi = {10.5281/zenodo.23664058},
  url = {https://zenodo.org/records/23664058},
  note = {CC BY 4.0},
}
RIS
TY  - UNPB
TI  - WardBench: Evaluating Policy Compliance, Dynamic FHIR Consent, and Construct Validity in Clinical EHR Agents
AU  - Vadim Nareyko
PY  - 2026
DA  - 2026/10/26
PB  - Zenodo
DO  - 10.5281/zenodo.23664058
UR  - https://zenodo.org/records/23664058
AB  - WardBench is a synthetic FHIR R4 agent benchmark for joint utility and policy compliance under role, HL7 PurposeOfUse, patient-stateful FHIR Consent, and PSY labels. Built on Synthea (seed 20261019): 120 main tasks (allowed, denied, boundary, adversarial), plus control and permitted twins, graded deterministically on trajectories (access, action, disclosure).  This draft reports a local 11-model evaluation (S1-S20b).  Draft status: held-out official evaluation is pending clinician/DPO review of the assumed policy table, results are not a legal compliance certification (HIPAA/GDPR/EHDS).  Code and review packet: https://github.com/WardBench/wardbench
ER  -