Leveraging Large Language Models for Temporal PHI De-identification in Real-World Clinical Notes
- 1Dell Medical School, The University of Texas at Austin
Abstract
De-identifying temporal protected health information (PHI) is challenging because temporal expressions appear in diverse formats and must be transformed while preserving clinically meaningful timelines. This study proposes and evaluates an LLM-based framework for temporal PHI de-identification in real-world clinical notes. Using 1,148 notes from 30 sarcoma patients (28,431 annotated temporal entities), we evaluated multiple modern LLMs for temporal entity extraction and surrogate generation. Proprietary models achieved strong extraction performance, with GPT-4o achieving the best overall results. Most models preserved temporal formatting (>99%), but surrogate generation remained challenging. GPT-5.4 achieved the highest shift correctness (90.2% on a shared-entity subset) and the lowest order violations. Error analysis revealed systematic shift deviations ({+/-}1, {+/-}30/31, {+/-}365 days), highlighting persistent limitations in LLM temporal reasoning. These findings suggest that reliable temporal de-identification will require hybrid or multi-agent approaches beyond standalone LLMs.
Related
- Evaluating and leveraging large language models in clinical pharmacology and therapeutics assessment: From exam takers to exam shapers
- Leveraging Large Language Models for Colorectal Cancer Symptom Extraction from MIMIC-IV Clinical Notes
- Bern Sleep-Wake Registry: temporal extraction of comorbidities from unstructured clinical text using a large language model
- Leveraging large language models to generate multiple-choice questions for epilepsy education
- Evaluation of the performance and temporal variability of large language models in patient education regarding pneumothorax: a seven-day analysis
BibTeX
@misc{wang2026leveraging,
title = {{Leveraging Large Language Models for Temporal PHI De-identification in Real-World Clinical Notes}},
author = {X. Wang and L. Wang and A. Wen and R. Li and S. Lu and Y. Hu and X. Li and H. Lyu and H. Liu},
howpublished = {medRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.22.26360316},
url = {https://www.medrxiv.org/content/10.64898/2026.09.22.26360316v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Leveraging Large Language Models for Temporal PHI De-identification in Real-World Clinical Notes
AU - X. Wang
AU - L. Wang
AU - A. Wen
AU - R. Li
AU - S. Lu
AU - Y. Hu
AU - X. Li
AU - H. Lyu
AU - H. Liu
PY - 2026
DA - 2026/09/22
PB - medRxiv
DO - 10.64898/2026.09.22.26360316
UR - https://www.medrxiv.org/content/10.64898/2026.09.22.26360316v1
AB - De-identifying temporal protected health information (PHI) is challenging because temporal expressions appear in diverse formats and must be transformed while preserving clinically meaningful timelines. This study proposes and evaluates an LLM-based framework for temporal PHI de-identification in real-world clinical notes. Using 1,148 notes from 30 sarcoma patients (28,431 annotated temporal entities), we evaluated multiple modern LLMs for temporal entity extraction and surrogate generation. Proprietary models achieved strong extraction performance, with GPT-4o achieving the best overall results. Most models preserved temporal formatting (>99%), but surrogate generation remained challenging. GPT-5.4 achieved the highest shift correctness (90.2% on a shared-entity subset) and the lowest order violations. Error analysis revealed systematic shift deviations ({+/-}1, {+/-}30/31, {+/-}365 days), highlighting persistent limitations in LLM temporal reasoning. These findings suggest that reliable temporal de-identification will require hybrid or multi-agent approaches beyond standalone LLMs.
ER -