Auditing LLM Chemical Priors for Multi-Objective Bayesian Optimization of Chemical Reactions: A Validity, Alignment, and Diversity Benchmark
- 1Independent Researcher
Abstract
Large language models (LLMs) are increasingly proposed as priors for cold-starting Bayesian optimization (BO) of chemical reactions, yet their benefit–risk profile in multi-objective reaction optimization remains poorly characterized. We present a systematic benchmark asking when LLM knowledge helps, is neutral, or hurts multi-objective BO (MOBO). Across seven public reaction benchmarks (6,384 candidate experiments), we compare nine strategies—including LLM-initialized BO, an LLM-only initializer, and established baselines (TSEMO, EDBO+-style scalarization, LLAMBO)—with 20 seeds per configuration (1,120 main-grid plus 360 ablation campaigns), scored by hypervolume completion and its sampling-efficiency integral (AUC), with paired Wilcoxon tests and two-family Benjamini–Hochberg correction against a random-init BO control. Three regimes emerge. On Suzuki–Miyaura, LLM initialization significantly improves sampling efficiency (ΔAUC +2.0 points, q = 0.026; alternative backbone +2.7 points, q = 1.4 × 10⁻⁴). On Buchwald–Hartwig, LLM priors are statistically indistinguishable from random. On a process-development coupling, LLM initialization significantly harms efficiency (ΔAUC −11.1 points, q = 0.0016) despite a 0% out-of-pool rate, whereas the Suzuki benefit coexists with a 45% out-of-pool rate—the prior’s direction, not its syntactic validity, controls the outcome. Diversity-based initialization (farthest point sampling) matches or exceeds LLM initialization wherever LLMs help, and no LLM-derived benefit exceeds what the knowledge-free diversity control already provides. A five-category diagnostic protocol intercepts 100% out-of-pool suggestions on the flow cases at zero experimental cost. LLM priors are not yet a reliable default for multi-objective reaction optimization; we provide a cost-free pre-campaign screening heuristic for deciding when to trust them.
Related
- Rescuable Stagnation: A Diagnostic for Benchmark Suitability in LLM-Guided Materials Optimization
- Quantifying Multi-Objective Optimization in Generative Chemistry for Molecule Design
- Electrochemical Platform for Automated Design-of-Experiment Optimization of Ni-Catalyzed Cross-Electrophile Coupling Reactions.
- Bi–O–S Catalyst for Enhanced CO2 Electroreduction to Formic Acid: Catalyst–Substrate Engineering and Multi-Objective Optimization
- Bi-objective optimization of simultaneous vibration suppression and energy harvesting in suspension systems based on mixed H2/H∞ control
BibTeX
@misc{zhang2026auditing,
title = {{Auditing LLM Chemical Priors for Multi-Objective Bayesian Optimization of Chemical Reactions: A Validity, Alignment, and Diversity Benchmark}},
author = {Fan Zhang},
howpublished = {ChemRxiv},
year = {2026},
month = sep,
doi = {10.26434/chemrxiv.15009537/v1},
url = {https://chemrxiv.org/doi/full/10.26434/chemrxiv.15009537/v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Auditing LLM Chemical Priors for Multi-Objective Bayesian Optimization of Chemical Reactions: A Validity, Alignment, and Diversity Benchmark
AU - Fan Zhang
PY - 2026
DA - 2026/09/28
PB - ChemRxiv
DO - 10.26434/chemrxiv.15009537/v1
UR - https://chemrxiv.org/doi/full/10.26434/chemrxiv.15009537/v1
AB - Large language models (LLMs) are increasingly proposed as priors for cold-starting Bayesian optimization (BO) of chemical reactions, yet their benefit–risk profile in multi-objective reaction optimization remains poorly characterized. We present a systematic benchmark asking when LLM knowledge helps, is neutral, or hurts multi-objective BO (MOBO). Across seven public reaction benchmarks (6,384 candidate experiments), we compare nine strategies—including LLM-initialized BO, an LLM-only initializer, and established baselines (TSEMO, EDBO+-style scalarization, LLAMBO)—with 20 seeds per configuration (1,120 main-grid plus 360 ablation campaigns), scored by hypervolume completion and its sampling-efficiency integral (AUC), with paired Wilcoxon tests and two-family Benjamini–Hochberg correction against a random-init BO control. Three regimes emerge. On Suzuki–Miyaura, LLM initialization significantly improves sampling efficiency (ΔAUC +2.0 points, q = 0.026; alternative backbone +2.7 points, q = 1.4 × 10⁻⁴). On Buchwald–Hartwig, LLM priors are statistically indistinguishable from random. On a process-development coupling, LLM initialization significantly harms efficiency (ΔAUC −11.1 points, q = 0.0016) despite a 0% out-of-pool rate, whereas the Suzuki benefit coexists with a 45% out-of-pool rate—the prior’s direction, not its syntactic validity, controls the outcome. Diversity-based initialization (farthest point sampling) matches or exceeds LLM initialization wherever LLMs help, and no LLM-derived benefit exceeds what the knowledge-free diversity control already provides. A five-category diagnostic protocol intercepts 100% out-of-pool suggestions on the flow cases at zero experimental cost. LLM priors are not yet a reliable default for multi-objective reaction optimization; we provide a cost-free pre-campaign screening heuristic for deciding when to trust them.
ER -