Answering clinicians' questions over trial evidence tables with verifiable, feedback-driven language models
- 1Arizona State University
Abstract
Systematic reviews condense clinical trials into evidence tables, yet clinicians can interrogate these tables only through database queries, and many questions concern attributes that the table does not record, such as a drug's target class or a harmonised endpoint. Here we introduce FD-SCoPE, a language-model framework that answers both kinds of question, exposes the query, the selected trials and the derivation rule behind every answer, and learns from expert corrections. On an oncology evidence table of 159 immune checkpoint inhibitor trial records, FD-SCoPE completed all 140 clinician-style tasks (alternatives, 90.7--97.9%). For questions needing derived attributes it retrieved 99.3% of relevant trial records at a positive predictive value of 89.8% and outperformed four alternative approaches (derived-value F1 77.7% versus 64.8--73.4%). Corrections on 299 questions, simulated from reference answers, raised F1 on 1,201 unseen questions from 77.9% to 84.9%. Language models coupled with executable queries, verified programs and expert feedback can give clinicians auditable access to trial evidence.
Related
- No Evidence of an Effect of a Single Mirror Visual Feedback Session on Subsequent Motor Imagery
- Patient experience in National digital health platforms: Evidence from patient-generated feedback in Saudi Arabia
- MEMOIR-VLM—a multimodal vision-language model for Alzheimer's disease classification and question answering
- S2F-Agent: Harnessing sequence-to-function models for verifiable genome interpretation
- Version-aware question answering over living oncology guidelines
BibTeX
@misc{choudhury2026answering,
title = {{Answering clinicians' questions over trial evidence tables with verifiable, feedback-driven language models}},
author = {M. Roy Choudhury and S. Roy Chowdhury and S. Sahoo and M. A. Khan and K. Z. R. Khakwani and M. B. Sonbol and I. Riaz and V. Gupta},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.01.756098},
url = {https://www.biorxiv.org/content/10.64898/2026.10.01.756098v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Answering clinicians' questions over trial evidence tables with verifiable, feedback-driven language models
AU - M. Roy Choudhury
AU - S. Roy Chowdhury
AU - S. Sahoo
AU - M. A. Khan
AU - K. Z. R. Khakwani
AU - M. B. Sonbol
AU - I. Riaz
AU - V. Gupta
PY - 2026
DA - 2026/10/01
PB - bioRxiv
DO - 10.64898/2026.10.01.756098
UR - https://www.biorxiv.org/content/10.64898/2026.10.01.756098v1
AB - Systematic reviews condense clinical trials into evidence tables, yet clinicians can interrogate these tables only through database queries, and many questions concern attributes that the table does not record, such as a drug's target class or a harmonised endpoint. Here we introduce FD-SCoPE, a language-model framework that answers both kinds of question, exposes the query, the selected trials and the derivation rule behind every answer, and learns from expert corrections. On an oncology evidence table of 159 immune checkpoint inhibitor trial records, FD-SCoPE completed all 140 clinician-style tasks (alternatives, 90.7--97.9%). For questions needing derived attributes it retrieved 99.3% of relevant trial records at a positive predictive value of 89.8% and outperformed four alternative approaches (derived-value F1 77.7% versus 64.8--73.4%). Corrections on 299 questions, simulated from reference answers, raised F1 on 1,201 unseen questions from 77.9% to 84.9%. Language models coupled with executable queries, verified programs and expert feedback can give clinicians auditable access to trial evidence.
ER -