Agents Propose, Code Verifies, Humans Decide: Verified Agentic Ingestion of Heterogeneous Scientific Tables into Relational Databases
Abstract
Public scientific datasets rarely arrive analysis-ready. They arrive as messy folders of wide matrices, long tables, spreadsheets, and per-cohort files whose keys and encodings differ, and every reanalysis must reconnect them first. A coding agent can automate this, but its mistakes can look plausible, and without an audit trail, a mistake that corrupts data can go uncaught. We present an AI-assisted ingestion pipeline built on one rule: LLM agents propose, deterministic code verifies, and humans decide. Narrow agents see compact, citable evidence about the files, never the data, and return typed proposals. A verifier with 39 check types tests each proposal against that evidence in two loops, one for the schema's structure and one for its meaning, and a human reviewer approves the schema and the load. Code compiles PostgreSQL DDL, reshapes the data, quarantines rows it cannot place, records provenance, and reconciles every staged row against the database. On four public biomedical datasets, one held out from development, 9 of 20 first proposals failed a check and every failure was caught: the loops repaired eight, and a one-line reviewer edit fixed the ninth. All 45.1 million values loaded with zero reconciliation gap. Discovering a dataset's structure took 4-7 LLM calls and preparing its data at most one more, costing under $0.50 per dataset. In a separate test, each of nine planted data errors was quarantined with a reason or recorded in an audit log.
Related
- ADMET-EvO: a self-evolving scientific agent for sustained research across heterogeneous tasks
- How Periodontal Disease Compromises Oral Functions Related to Ingestion: Evidence From Systematic Review and Meta‐Analysis
- A Preliminary Study on Polyurethane Foam Ingestion by Tenebrio molitor Larvae and Potential Effects of Depolymerising Bacterial Supplementation
- Biochar Ingestion Is Associated with Hindgut Symbiotic Disruption and Mortality in Coptotermes formosanus: A Proposed Protist-to-Bacterial Cascade
- The use of sequence databases for taxonomic research in phytovirology and the application of factor analysis of correspondences to the clustering of geminiviruses
BibTeX
@misc{rankothge2026agents,
title = {{Agents Propose, Code Verifies, Humans Decide: Verified Agentic Ingestion of Heterogeneous Scientific Tables into Relational Databases}},
author = {Bharatha Rankothge and Suzanne Hodgkins and Virginia Rich and Michael Kirby},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.11045},
eprint = {2611.11045},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.11045},
}
RIS
TY - UNPB
TI - Agents Propose, Code Verifies, Humans Decide: Verified Agentic Ingestion of Heterogeneous Scientific Tables into Relational Databases
AU - Bharatha Rankothge
AU - Suzanne Hodgkins
AU - Virginia Rich
AU - Michael Kirby
PY - 2026
DA - 2026/11/08
PB - arXiv
DO - 10.48550/arxiv.2611.11045
UR - https://arxiv.org/abs/2611.11045
AB - Public scientific datasets rarely arrive analysis-ready. They arrive as messy folders of wide matrices, long tables, spreadsheets, and per-cohort files whose keys and encodings differ, and every reanalysis must reconnect them first. A coding agent can automate this, but its mistakes can look plausible, and without an audit trail, a mistake that corrupts data can go uncaught. We present an AI-assisted ingestion pipeline built on one rule: LLM agents propose, deterministic code verifies, and humans decide. Narrow agents see compact, citable evidence about the files, never the data, and return typed proposals. A verifier with 39 check types tests each proposal against that evidence in two loops, one for the schema's structure and one for its meaning, and a human reviewer approves the schema and the load. Code compiles PostgreSQL DDL, reshapes the data, quarantines rows it cannot place, records provenance, and reconciles every staged row against the database. On four public biomedical datasets, one held out from development, 9 of 20 first proposals failed a check and every failure was caught: the loops repaired eight, and a one-line reviewer edit fixed the ninth. All 45.1 million values loaded with zero reconciliation gap. Discovering a dataset's structure took 4-7 LLM calls and preparing its data at most one more, costing under $0.50 per dataset. In a separate test, each of nine planted data errors was quarantined with a reason or recorded in an audit log.
ER -