Debate, Not Voting: Agentic Arbitration over Weak Optical Chemical Structure Recognition Models
Abstract
Optical chemical structure recognition (OCSR) turns a picture of a molecule into a machine-readable string, and it stands between decades of printed chemistry and any computational use. On real documents individual models remain unreliable. Over a 1000-molecule sample of MolParser, three readers score 46.0%, 43.4% and 42.4% exact match, yet at least one of them is right on 60.8% of molecules. That gap is an ensembling problem, but not one voting can solve, because the readers seldom agree and seldom err together. We place the readers inside a multi-agent scaffold. Four language-model agents with cheminformatics tools argue about a candidate structure, write findings to an append-only knowledge base, and are held to render-and-compare gates that admit a correction only when it is demonstrably better. The debate reaches 61.9%, beating the best single reader by $15.9$ points and plurality voting by $9.2$ points (p<10−9 for both, robust to correction for all $27$ tests we report), and is indistinguishable from the any-reader-right ceiling. That aggregate hides two opposing effects. On the $661$ molecules with no stereochemistry the debate is $3.9$ points above the ceiling (p=0.004), constructing structures no reader proposed, while on the $339$ stereochemical ones it falls below. We then swap the arbitrating model. A stronger arbitrator adds $2.2$ points (nominal p=0.039; neither this nor the +3.9 survives Bonferroni correction over all $27$ tests, so both are suggestive), and its mechanism is visible: it opens a correction on $423$ of $1000$ molecules against $120$, and an interaction test indicates it depends less on the readers (nominal p=0.018, not correction-robust). We conclude that the useful unit of progress is the arbitration policy rather than the reader, and that OCSR accuracy is better read against the union oracle than against the best single model.
Related
- Harness-Driven Agentic Operation of Optical Pooled Screen Analysis
- Competing Hypotheses and Scientific Non-Recognition: Investigating the Hector’s and Māui Dolphin Decline in the Context of Wartime Chemical Legacies
- Mate recognition in orchid bees is mediated by acquired chemical signals.
- Selective Recognition, Covalent Linkage and Chemical Exchange of Artificial Cells through Engineered α-Hemolysin
- Agentic cryo-EM structure determination with CryoPilot
BibTeX
@misc{chauvin2026debate,
title = {{Debate, Not Voting: Agentic Arbitration over Weak Optical Chemical Structure Recognition Models}},
author = {Paul Chauvin and Sven Jager},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.33002},
eprint = {2610.33002},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.33002},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Debate, Not Voting: Agentic Arbitration over Weak Optical Chemical Structure Recognition Models
AU - Paul Chauvin
AU - Sven Jager
PY - 2026
DA - 2026/10/06
PB - arXiv
DO - 10.48550/arxiv.2610.33002
UR - https://arxiv.org/abs/2610.33002
AB - Optical chemical structure recognition (OCSR) turns a picture of a molecule into a machine-readable string, and it stands between decades of printed chemistry and any computational use. On real documents individual models remain unreliable. Over a 1000-molecule sample of MolParser, three readers score 46.0%, 43.4% and 42.4% exact match, yet at least one of them is right on 60.8% of molecules. That gap is an ensembling problem, but not one voting can solve, because the readers seldom agree and seldom err together. We place the readers inside a multi-agent scaffold. Four language-model agents with cheminformatics tools argue about a candidate structure, write findings to an append-only knowledge base, and are held to render-and-compare gates that admit a correction only when it is demonstrably better. The debate reaches 61.9%, beating the best single reader by $15.9$ points and plurality voting by $9.2$ points (p<10^−9 for both, robust to correction for all $27$ tests we report), and is indistinguishable from the any-reader-right ceiling. That aggregate hides two opposing effects. On the $661$ molecules with no stereochemistry the debate is $3.9$ points above the ceiling (p=0.004), constructing structures no reader proposed, while on the $339$ stereochemical ones it falls below. We then swap the arbitrating model. A stronger arbitrator adds $2.2$ points (nominal p=0.039; neither this nor the +3.9 survives Bonferroni correction over all $27$ tests, so both are suggestive), and its mechanism is visible: it opens a correction on $423$ of $1000$ molecules against $120$, and an interaction test indicates it depends less on the readers (nominal p=0.018, not correction-robust). We conclude that the useful unit of progress is the arbitration policy rather than the reader, and that OCSR accuracy is better read against the union oracle than against the best single model.
ER -