Planner-Executor Style Multimodal Agentic System to Answer Patient Questions in Lung Cancer Screening CT
- 1University of California, San Francisco
Abstract
BackgroundAccurate patient understanding of lung cancer screening (LCS) results is critical for engagement and follow-up adherence, given the currently low screening uptake. While artificial intelligence (AI) systems show promise in facilitating patient communication, its effective use in clinical settings requires appropriate invocation of imaging tools and self-regulation by deferring certain questions to physicians. To this end, we developed and evaluated a planner-executor style multimodal agentic system to answer simulated patient questions about LCS CT results.
MethodsThis retrospective study utilized 116 LCS CT reports and images collected from a tertiary academic hospital. The system employed a multi-agent architecture (planner and executor) to mimic clinical reasoning. We developed a large language model (LLM)-based question generation framework to prepare a comprehensive set of 699 simulated patient questions, categorized as answerable and defer-to-doctor to assess self-regulation. Performance was evaluated on tool-calling, self-regulation accuracy (ability to correctly defer out-of-scope questions to a human provider) assessed by LLM-as-a-judge, and clinical quality assessed by a reader performance study with four physicians evaluating a subset of 100 responses.
ResultsThe system demonstrated high overall tool-calling accuracy of 93.1% (646/694; 95% CI: 91.2%, 95.0%) and self-regulation accuracy of 92.6% (462/499; 90.2%, 94.8%). For answerable questions, the percentage of responses receiving perfect 5.0 scores across all readers included 78.9% (180/228) for clinical accuracy (inter-rater agreement: Gwets AC2=0.92), and 73.2% (167/228) for patient understandability (0.86). For defer-to-doctor questions, the percentage of responses included 77.3% (133/172) for clinical accuracy (0.80) and 70.3% (121/172) for patient understandability (0.78).
ConclusionsThe planner-executor style multi-modal agentic system reliably answers patient-specific questions about LCS. Its high tool-calling and self-regulation performance demonstrates its potential as a safe and effective digital communication facilitator in LCS programs.
Related
- An auditable multimodal agentic system for sequential decision-making across the acute stroke pathway
- COMBINING CHEST X-RAY AND CLINICAL COVARIATES FOR OSTEOPOROSIS SCREENING: A MULTIMODAL DEEP LEARNING APPROACH
- A multimodal precision oncology platform integrating genomics,single-cell transcriptomics, and functional drug screening expandstherapeutic options in pediatric brain tumors
- Artificial Intelligence in Ocular Surface Oncology: A Narrative Review of Multimodal Imaging, Smartphone-Based Screening, and Clinical Decision Support
- Quality, Readability, Understandability, and Actionability of AI Chatbots’ Responses to Dental Patient Questions: A Comparative Evaluation
BibTeX
@misc{sakiyama2026plannerexecutor,
title = {{Planner-Executor Style Multimodal Agentic System to Answer Patient Questions in Lung Cancer Screening CT}},
author = {K. Sakiyama and A. Serapio and M. Ye and A. Nowroozi and M. Bondarenko and Y. Wu and K. Qi and J. Liu and M. Vella and Y. Yu and T. Schnitzler and A. S. Rustagi and J. H. Sohn},
howpublished = {medRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.02.26364620},
url = {https://www.medrxiv.org/content/10.64898/2026.10.02.26364620v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Planner-Executor Style Multimodal Agentic System to Answer Patient Questions in Lung Cancer Screening CT
AU - K. Sakiyama
AU - A. Serapio
AU - M. Ye
AU - A. Nowroozi
AU - M. Bondarenko
AU - Y. Wu
AU - K. Qi
AU - J. Liu
AU - M. Vella
AU - Y. Yu
AU - T. Schnitzler
AU - A. S. Rustagi
AU - J. H. Sohn
PY - 2026
DA - 2026/10/02
PB - medRxiv
DO - 10.64898/2026.10.02.26364620
UR - https://www.medrxiv.org/content/10.64898/2026.10.02.26364620v1
AB - BackgroundAccurate patient understanding of lung cancer screening (LCS) results is critical for engagement and follow-up adherence, given the currently low screening uptake. While artificial intelligence (AI) systems show promise in facilitating patient communication, its effective use in clinical settings requires appropriate invocation of imaging tools and self-regulation by deferring certain questions to physicians. To this end, we developed and evaluated a planner-executor style multimodal agentic system to answer simulated patient questions about LCS CT results. MethodsThis retrospective study utilized 116 LCS CT reports and images collected from a tertiary academic hospital. The system employed a multi-agent architecture (planner and executor) to mimic clinical reasoning. We developed a large language model (LLM)-based question generation framework to prepare a comprehensive set of 699 simulated patient questions, categorized as answerable and defer-to-doctor to assess self-regulation. Performance was evaluated on tool-calling, self-regulation accuracy (ability to correctly defer out-of-scope questions to a human provider) assessed by LLM-as-a-judge, and clinical quality assessed by a reader performance study with four physicians evaluating a subset of 100 responses. ResultsThe system demonstrated high overall tool-calling accuracy of 93.1% (646/694; 95% CI: 91.2%, 95.0%) and self-regulation accuracy of 92.6% (462/499; 90.2%, 94.8%). For answerable questions, the percentage of responses receiving perfect 5.0 scores across all readers included 78.9% (180/228) for clinical accuracy (inter-rater agreement: Gwets AC2=0.92), and 73.2% (167/228) for patient understandability (0.86). For defer-to-doctor questions, the percentage of responses included 77.3% (133/172) for clinical accuracy (0.80) and 70.3% (121/172) for patient understandability (0.78). ConclusionsThe planner-executor style multi-modal agentic system reliably answers patient-specific questions about LCS. Its high tool-calling and self-regulation performance demonstrates its potential as a safe and effective digital communication facilitator in LCS programs.
ER -