{
  "abstract": "ObjectivesStructuring oncology clinical notes into registry-grade variables is essential for research and care but remains labour-intensive and error-prone. This study developed and evaluated a privacy-preserving large language model pipeline for oncology registry abstraction in a real-world clinical setting.\n\nMethodsAn open-source Meta Llama 3.3 70B-based pipeline, prompted without task-specific fine-tuning, was deployed to extract over 50 variables from 6,700 oncology notes at a cancer centre in Singapore. Data were de-identified locally using a Hide-In-Plain-Sight approach, ensuring no identifiable data left hospital infrastructure. Performance was assessed on 200 randomly sampled notes with adjudicated ground truth, using a structure-aware framework that classified outputs as correct, missing, spurious, or incorrect.\n\nResultsF1 scores were high across variables, including diagnosis (97.2%), histology (95.8%), stage (92.6%), biomarkers (91.4%), and treatments (88.1%). Preliminary transferability testing on 50 external notes showed strong performance for core diagnostic variables, with reduced performance for list-valued variables.\n\nDiscussionThe structure-aware evaluation approach separated omissions from fabrications, providing more clinically actionable insight than aggregate metrics alone.\n\nConclusionPrivacy-preserving large language models achieved high extraction accuracy under a clinician-verified reference standard, supporting their feasibility for oncology registry abstraction in real-world clinical settings.\n\nWhat is already known on this topicStructuring oncology clinical notes into registry-grade variables is essential for cancer research and care but remains labour-intensive, costly, and inconsistent from one reviewer to the next when performed manually. Large language models offer potential to automate this task, but their use in real-world registries has been limited by concerns over patient data privacy, reliance on task-specific fine-tuning, and evaluation methods that obscure the difference between missing and fabricated information.\n\nWhat this study addsThis study shows that an open-source, privacy-preserving large language model pipeline, deployed entirely within hospital infrastructure without task-specific fine-tuning, can extract over 50 registry variables from oncology notes with high accuracy (F1 scores 88-97% across key variables), validated against ground truth checked by two clinicians. Using a structure-aware evaluation framework, we demonstrate that errors can be meaningfully separated into missing, spurious, and incorrect categories, offering clearer insight into model behaviour than aggregate accuracy metrics alone, and show preliminary evidence of transferability to notes from an external setting.\n\nHow this study might affect research, practice or policyThese findings support the feasibility of deploying privacy-preserving, locally hosted large language models for oncology registry abstraction without requiring identifiable data to leave hospital systems or extensive model customisation. The structure-aware evaluation framework is also shared as a reusable template, enabling other institutions to validate similar AI extraction tools in a consistent, clinically meaningful way before wider implementation.",
  "abstract_shown": true,
  "affiliations": [
    "Bond University"
  ],
  "article_type": "PUBLISHAHEADOFPRINT",
  "arxiv": null,
  "author_total": 9,
  "authors": [
    {
      "name": "M. Moldovan"
    },
    {
      "name": "R. Enikeev"
    },
    {
      "name": "M. Chu"
    },
    {
      "name": "A. Amalraj"
    },
    {
      "name": "P. P. Koli"
    },
    {
      "name": "S. S. Abdul"
    },
    {
      "name": "H. Sivaraj"
    },
    {
      "affs": [
        0
      ],
      "name": "U. Iqbal"
    },
    {
      "name": "C. K. Toh"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/8g077f0thn431.bib",
  "cited_by": 0,
  "doi": "10.64898/2026.05.18.26353541",
  "fulltext_url": "https://www.medrxiv.org/content/10.64898/2026.05.18.26353541v2.full",
  "id": "8g077f0thn431",
  "issns": [],
  "landing_url": "https://www.medrxiv.org/content/10.64898/2026.05.18.26353541v2",
  "language": null,
  "licence": "CC-BY-NC",
  "licence_label": "CC BY-NC",
  "licence_source": "article",
  "licence_url": "https://creativecommons.org/licenses/by-nc/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "medrxiv",
      "health informatics"
    ],
    [
      "openalex",
      "1702|Machine Learning in Healthcare"
    ]
  ],
  "openalex": "W7162039516",
  "pdf_url": "https://www.medrxiv.org/content/10.64898/2026.05.18.26353541v2.full.pdf",
  "pmcid": null,
  "pmid": null,
  "published": "2026-05-18",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/8g077f0thn431.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:21:16Z",
      "native_id": "10.64898/2026.05.18.26353541",
      "source": "medrxiv",
      "url": "https://www.medrxiv.org/content/10.64898/2026.05.18.26353541v3"
    }
  ],
  "subjects": [
    {
      "field": "medicine",
      "origin": "medrxiv:health informatics",
      "subfield": "epidemiology-public-health"
    }
  ],
  "title": "Privacy-Preserving Large Language Model Deployment for Oncology Registry Abstraction: Structure-Aware Evaluation in a Real-World Clinical Setting",
  "title_text": "Privacy-Preserving Large Language Model Deployment for Oncology Registry Abstraction: Structure-Aware Evaluation in a Real-World Clinical Setting",
  "type": "preprint",
  "updated_at": "2026-10-30T12:21:21Z",
  "url": "https://research.draconitic.com/paper/8g077f0thn431",
  "venue": "medRxiv",
  "versions": []
}