From Clinical Free Text to Auditable Concepts: An Agentic Framework for Interpretable Prediction
- 1Vanderbilt University Medical Center
Abstract
Across application domains, predictive signals often sit in unstructured free text rather than structured fields, yet turning that text into useful and interpretable features is difficult. Running large language models (LLMs) over an entire corpus is costly and hard to reproduce, while end-to-end text representations can rely on surface cues that are difficult to inspect. We present an agentic workflow that takes a prediction task and a raw text corpus as input and produces an auditable feature layer. The first two agents use an LLM to derive a task-specific predictor taxonomy and weakly label a bounded text sample; routed local extractors then process the corpus, and a deterministic builder aggregates the evidence into a dynamic, longitudinal concept bottleneck. We evaluate the framework on medication discontinuation in a longitudinal oncology cohort and 30-day readmission in MIMIC-IV. With gradient boosting, the longitudinal bottleneck increases area under the receiver operating characteristic curve (AUROC) over coarse concept buckets from 0.700 to 0.761 for medication discontinuation and from 0.576 to 0.609 for readmission. The proposed framework achieves predictive performance comparable to direct BioClinicalBERT prediction on both tasks while additionally providing explicit, interpretable, and traceable task-specific concepts. LLM use is confined to a bounded weak-labeling stage costing $24.00 and $23.39, respectively, compared with projected costs of $10,648 and $11,519 for exhaustive sentence-level LLM processing of the full corpora, demonstrating the substantial cost efficiency of the proposed agentic system.
Related
- KT-CBHNET: a spatio-temporal deep learning framework for interpretable ECG signal classification for cardiovascular disease prediction
- HAFI-DTA: an interpretable multimodal framework for drug-target affinity prediction based on hierarchical attention guidance and feature iterative interaction
- A leakage-aware and interpretable machine learning framework for ovarian cancer prediction using the PLCO screening dataset
- From Nutritional Epidemiology to Clinical Prediction: Interpretable Machine Learning Reveals Subtle Associations Between Dietary Antioxidant Profiles and Non-Melanoma Skin Cancer in Older Adults.
- Lung cancer risk prediction using interpretable ensemble models on lifestyle and clinical data
BibTeX
@misc{ni2026from,
title = {{From Clinical Free Text to Auditable Concepts: An Agentic Framework for Interpretable Prediction}},
author = {C. Ni and W. Liu and Q. Song and M. Murrow and B. A. Malin and Z. Yin},
howpublished = {medRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.02.26362086},
url = {https://www.medrxiv.org/content/10.64898/2026.09.02.26362086v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - From Clinical Free Text to Auditable Concepts: An Agentic Framework for Interpretable Prediction
AU - C. Ni
AU - W. Liu
AU - Q. Song
AU - M. Murrow
AU - B. A. Malin
AU - Z. Yin
PY - 2026
DA - 2026/09/02
PB - medRxiv
DO - 10.64898/2026.09.02.26362086
UR - https://www.medrxiv.org/content/10.64898/2026.09.02.26362086v1
AB - Across application domains, predictive signals often sit in unstructured free text rather than structured fields, yet turning that text into useful and interpretable features is difficult. Running large language models (LLMs) over an entire corpus is costly and hard to reproduce, while end-to-end text representations can rely on surface cues that are difficult to inspect. We present an agentic workflow that takes a prediction task and a raw text corpus as input and produces an auditable feature layer. The first two agents use an LLM to derive a task-specific predictor taxonomy and weakly label a bounded text sample; routed local extractors then process the corpus, and a deterministic builder aggregates the evidence into a dynamic, longitudinal concept bottleneck. We evaluate the framework on medication discontinuation in a longitudinal oncology cohort and 30-day readmission in MIMIC-IV. With gradient boosting, the longitudinal bottleneck increases area under the receiver operating characteristic curve (AUROC) over coarse concept buckets from 0.700 to 0.761 for medication discontinuation and from 0.576 to 0.609 for readmission. The proposed framework achieves predictive performance comparable to direct BioClinicalBERT prediction on both tasks while additionally providing explicit, interpretable, and traceable task-specific concepts. LLM use is confined to a bounded weak-labeling stage costing $24.00 and $23.39, respectively, compared with projected costs of $10,648 and $11,519 for exhaustive sentence-level LLM processing of the full corpora, demonstrating the substantial cost efficiency of the proposed agentic system.
ER -