No Value Is Better Than a Wrong Value: Finding and Handling Erroneous Data in Literature-Mined Datasets for AI
Abstract
Machine learning across materials science, chemistry, energy, and medicine increasingly relies on literature-extracted datasets due to a lack of curated databases. However, automated extraction introduces errors: text recognition misreads digits, units and notations are confused, values are misattributed, and LLMs hallucinate numbers. Drawing on our work building an electro-optic property database, we present a taxonomy of errors organized by their origin and signatures, alongside a layered framework for error detection. The framework combines schema and provenance checks, physical admissibility (bounds, symmetry, and governing equations), cross-source statistical consensus, model auditing, and suspicion-ranked review. In a production run, our verification gate halted 402 of 816 candidate values with typed reasons. Yet, an audit against curated references showed that 30 of 112 accepted values were defective; standard symmetry rules alone caught 21 of these. On measured BaTiO3 and KNbO3 tensors, constitutive identities caught 8 of 10 single-value errors and isolated the corrupted entries. In a controlled benchmark, the combined framework caught 81% of errors at 85% precision, including errors in single-report materials, bringing model accuracy on held-out data to within 3% of a model trained on clean data. Uncleaned data severely impairs discovery: 89% of the twenty largest values were errors, causing AI searches to prioritize false positives. Once bad numbers are controlled, missing measurement conditions emerge as the main limit, as distinct records become indistinguishable without full context. We outline directions for the field, including shared physics rule libraries and self-reporting databases.
Related
- INI-VPINN: A Variational Physics-Informed Neural Network with Implicit Neumann and Interface Handling for Multi-Material Domains with Geometric Singularities
- Handling geometric singularities efficiently with the immersed-boundary method: application to riblets
- Physics-Informed Method of Group Data Handling: Adaptive Construction of Functional Representations with an Application to the Navier-Stokes Equations
- Characteristic boundary conditions using ghost cells (GC-CBC) for high-order finite volume simulations of multicomponent Euler gas equations handling near-zero/sonic velocities
- Do Better Scores Mean Better Physics? Physics-Grounded Explanations for Sim2Real Neural Operators
BibTeX
@misc{manoj2026value,
title = {{No Value Is Better Than a Wrong Value: Finding and Handling Erroneous Data in Literature-Mined Datasets for AI}},
author = {Achyut Manoj and Advai Aiyer and Andrew Lee and Nayan Saini and Pragya Birla and Dibakar Datta},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.34642},
eprint = {2610.34642},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.34642},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - No Value Is Better Than a Wrong Value: Finding and Handling Erroneous Data in Literature-Mined Datasets for AI
AU - Achyut Manoj
AU - Advai Aiyer
AU - Andrew Lee
AU - Nayan Saini
AU - Pragya Birla
AU - Dibakar Datta
PY - 2026
DA - 2026/10/25
PB - arXiv
DO - 10.48550/arxiv.2610.34642
UR - https://arxiv.org/abs/2610.34642
AB - Machine learning across materials science, chemistry, energy, and medicine increasingly relies on literature-extracted datasets due to a lack of curated databases. However, automated extraction introduces errors: text recognition misreads digits, units and notations are confused, values are misattributed, and LLMs hallucinate numbers. Drawing on our work building an electro-optic property database, we present a taxonomy of errors organized by their origin and signatures, alongside a layered framework for error detection. The framework combines schema and provenance checks, physical admissibility (bounds, symmetry, and governing equations), cross-source statistical consensus, model auditing, and suspicion-ranked review. In a production run, our verification gate halted 402 of 816 candidate values with typed reasons. Yet, an audit against curated references showed that 30 of 112 accepted values were defective; standard symmetry rules alone caught 21 of these. On measured BaTiO3 and KNbO3 tensors, constitutive identities caught 8 of 10 single-value errors and isolated the corrupted entries. In a controlled benchmark, the combined framework caught 81% of errors at 85% precision, including errors in single-report materials, bringing model accuracy on held-out data to within 3% of a model trained on clean data. Uncleaned data severely impairs discovery: 89% of the twenty largest values were errors, causing AI searches to prioritize false positives. Once bad numbers are controlled, missing measurement conditions emerge as the main limit, as distinct records become indistinguishable without full context. We outline directions for the field, including shared physics rule libraries and self-reporting databases.
ER -