{ "abstract": "Background\n\nThe increasing availability of machine-readable research data has created a growing need for efficient large-scale data harmonization (DH). Although large language models (LLMs) show promise for reducing the time and labor required for DH, their effective integration into harmonization workflows remains an important methodological challenge.Objective\n\nThis study aims to develop and evaluate a semiautomated, human-in-the-loop (HITL) LLM-assisted DH workflow as a proof of concept across 8 projects within a research consortium focused on disparities in multiple chronic conditions.Methods\n\nWe developed an LLM-assisted DH workflow that combined preprocessing, semantic mapping, variable response mapping, synthetic data generation, data transformation, and iterative researcher review. Using Generative Pretrained Transformer (GPT-4o) hosted in a secure academic environment, project-specific survey items were mapped to 114 Consortium Common Data Element (CDE) semantic groups. Semantic mapping accuracy was evaluated based on the Consortium's consensus harmonization results. Complete and incomplete semantic mappings were distinguished according to construct overlap, and incomplete mappings requiring context-dependent human judgment were excluded for mapped item pairs and CDE semantic groups with and without HITL review.Results\n\nFollowing preprocessing, 884 survey items were included in the DH workflow, of which 812 were mapped to a mean of 79 CDE semantic groups per project. Mean semantic mapping accuracy for item pairs was 95.9% (SD 5.7%) with HITL review and 86.7% (SD 8.6%) without HITL review. At the CDE semantic-group level, the corresponding accuracies were 98.9% (SD 1.7%) and 94.9% (SD 4.0%), respectively. Across projects, omission of valid semantic mappings occurred more frequently than incorrect semantic mappings, particularly for heterogeneous response structures and subjective constructs measured using different instruments. Pipeline components-including concept-based preprocessing, iterative prompt refinement, and targeted HITL review-improved mapping completeness and supported variable harmonization across heterogeneous datasets.Conclusions\n\nThis proof-of-concept study demonstrates that the effectiveness of LLM-assisted DH depends more on the design of a structured HITL workflow rather than on the LLM alone. Combining preprocessing, iterative verification, and targeted human oversight improved the efficiency and reliability of DH while preserving researcher judgment for complex mapping decisions.", "abstract_shown": true, "affiliations": [ "Department of Epidemiology and Biostatistics, University of California, 550 16th St., Floor 2, San Francisco, CA, 94143, United States.", "Division of Preventive Science, Department of Medicine, University of California, San Francisco, CA, United States.", "Research Coordinating Center for Reducing Disparities in Multiple Chronic Diseases, University of California, San Francisco, CA, United States.", "Computational Precision Health, University of California San Francisco and University of California Berkeley, Berkeley, CA, United States.", "Social and Behavioral Sciences Branch, Division of Intramural Research, Eunice Kennedy Shriver National Institute of Child Health and Human Development, BG 6710 Rockledge Dr Wing B, Bethesda, MD, 20817, United States, 1 301-555-6677.", "Division of Oral Epidemiology & Dental Public Health, University of California, San Francisco, CA, United States.", "Pharmaceutical Science and Pharmacogenomics program, University of California, San Francisco, CA, United States.", "Bakar Computational Health Sciences Institute, University of California, San Francisco, CA, United States.", "College of Computer Mathematical and Natural Science, Department of Epidemiology and Biostatistics, School of Public Health, The Artificial Intelligence Interdisciplinary Institute at Maryland (AIM), University System of Maryland, College Park, MD, United States.", "Francis I. Proctor Foundation, University of California, San Francisco, CA, United States.", "Massey Comprehensive Cancer Center, Virginia Commonwealth University, Richmond, VA, United States." ], "article_type": "research article", "arxiv": null, "author_total": 14, "authors": [ { "affs": [ 0, 1, 2, 3, 4 ], "name": "Hyelee Kim", "orcid": "0000-0001-5170-4057" }, { "affs": [ 2, 5 ], "name": "Shuang Liang", "orcid": "0009-0000-5142-2703" }, { "affs": [ 2, 5 ], "name": "Kathy Lanier", "orcid": "0000-0001-6482-4509" }, { "affs": [ 2, 5 ], "name": "Sarit Helman", "orcid": "0009-0001-4152-9555" }, { "affs": [ 2, 5 ], "name": "Nancy F Cheng", "orcid": "0000-0002-9902-0997" }, { "affs": [ 2, 5 ], "name": "Niloufar Ameli" }, { "affs": [ 6, 7 ], "name": "Jaysón Davidson", "orcid": "0000-0001-9066-9872" }, { "affs": [ 0, 1 ], "name": "Shivani Mehta", "orcid": "0000-0002-1417-6694" }, { "affs": [ 0, 3, 7, 8 ], "name": "Yulin Hswen", "orcid": "0000-0003-3203-1322" }, { "affs": [ 0, 9 ], "name": "Catherine E Oldenburg", "orcid": "0000-0002-0763-399X" }, { "affs": [ 0, 2, 5, 10 ], "name": "Kim F Rhoads", "orcid": "0000-0002-2558-6509" }, { "affs": [ 1, 2 ], "name": "Edwin D Charlebois", "orcid": "0000-0001-7237-2119" }, { "affs": [ 2, 5, 7 ], "name": "Stuart A Gansky", "orcid": "0000-0003-3147-0884" }, { "affs": [ 0, 1, 2, 3, 7 ], "name": "William Brown Iii", "orcid": "0000-0003-4322-4859" } ], "bibtex": "https://research.draconitic.com/paper/emttkfvra1xr9.bib", "cited_by": 0, "doi": "10.2196/93996", "fulltext_url": "https://europepmc.org/articles/PMC13690333", "id": "emttkfvra1xr9", "issns": [ "2817-1705" ], "landing_url": "https://europepmc.org/article/MED/42912962", "language": "eng", "licence": "CC-BY", "licence_label": "CC BY", "licence_source": "article", "licence_url": "https://creativecommons.org/licenses/by/4.0/", "metadata_licence": null, "native_subjects": [ [ "openalex", "1710|Research Data Management Practices" ] ], "openalex": "W7204913484", "pdf_url": "https://europepmc.org/articles/PMC13690333?pdf=render", "pmcid": "PMC13690333", "pmid": "42912962", "published": "2026-10-22", "publisher": null, "ris": "https://research.draconitic.com/paper/emttkfvra1xr9.ris", "sources": [ { "first_seen": "2026-10-30T12:20:10Z", "native_id": "MED:42912962", "source": "europepmc", "url": "https://europepmc.org/article/MED/42912962" } ], "subjects": [], "title": "Large Language Model-Assisted Semiautomated Data Harmonization Pipeline for the Multiple Chronic Disease Disparities Research Consortium: Framework Development and Comparative Validation Study.", "title_text": "Large Language Model-Assisted Semiautomated Data Harmonization Pipeline for the Multiple Chronic Disease Disparities Research Consortium: Framework Development and Comparative Validation Study.", "type": "peer-reviewed", "updated_at": "2026-10-30T12:20:10Z", "url": "https://research.draconitic.com/paper/emttkfvra1xr9", "venue": "JMIR AI", "versions": [] }