Version-aware question answering over living oncology guidelines
- 1Arizona State University
Abstract
Living clinical guidelines revise recommendations as evidence emerges, so an AI answer can be faithful to a real guideline and still be out of date. Here we examine this failure using ASCOBench, 288 unique questions in 96 three-turn conversations grounded in versioned American Society of Clinical Oncology (ASCO) breast and prostate cancer guidelines, with oncologist-reviewed reference answers. Oncologists corrected one in four version-sensitive answers drafted by a frontier model, against fewer than one in ten factual ones. Across the guideline corpus, 19 recommendations changed between versions, seven of them reversals, yet only 1 of 14 superseded documents states that it has been replaced. With a strong model, retrieval over several guideline versions made stale answers four times more frequent than answering without retrieval. SentryLine, a verification-first system, lowered incorrect answers to 4.2-5.2% from 9.7-22.9% for baselines across three models. Recognizing change before a guideline declares it remains an open problem.
Related
- MEMOIR-VLM—a multimodal vision-language model for Alzheimer's disease classification and question answering
- LiteMedCoT-VL: Parameter-Efficient Adaptation for Medical Visual Question Answering
- Relation-aware dynamic knowledge subgraph enhanced large language models for medical multiple-choice question answering
- Entity-centric evaluation of large language model responses for medical question-answering tasks
- Answering clinicians' questions over trial evidence tables with verifiable, feedback-driven language models
BibTeX
@misc{kumar2026versionaware,
title = {{Version-aware question answering over living oncology guidelines}},
author = {T. Ravi Kumar and A. I. Kaniyar Narayana Iyengar and M. A. Khan and K. Z. R. Khakwani and K. Kathuria and M. B. Sonbol and I. Riaz and V. Gupta},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.04.756483},
url = {https://www.biorxiv.org/content/10.64898/2026.10.04.756483v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Version-aware question answering over living oncology guidelines
AU - T. Ravi Kumar
AU - A. I. Kaniyar Narayana Iyengar
AU - M. A. Khan
AU - K. Z. R. Khakwani
AU - K. Kathuria
AU - M. B. Sonbol
AU - I. Riaz
AU - V. Gupta
PY - 2026
DA - 2026/10/04
PB - bioRxiv
DO - 10.64898/2026.10.04.756483
UR - https://www.biorxiv.org/content/10.64898/2026.10.04.756483v1
AB - Living clinical guidelines revise recommendations as evidence emerges, so an AI answer can be faithful to a real guideline and still be out of date. Here we examine this failure using ASCOBench, 288 unique questions in 96 three-turn conversations grounded in versioned American Society of Clinical Oncology (ASCO) breast and prostate cancer guidelines, with oncologist-reviewed reference answers. Oncologists corrected one in four version-sensitive answers drafted by a frontier model, against fewer than one in ten factual ones. Across the guideline corpus, 19 recommendations changed between versions, seven of them reversals, yet only 1 of 14 superseded documents states that it has been replaced. With a strong model, retrieval over several guideline versions made stale answers four times more frequent than answering without retrieval. SentryLine, a verification-first system, lowered incorrect answers to 4.2-5.2% from 9.7-22.9% for baselines across three models. Recognizing change before a guideline declares it remains an open problem.
ER -