MolP2SBench: Do LLMs Really Reason from Molecular Properties to Structure?
Abstract
Large language models (LLMs) are increasingly used in molecular discovery and structure elucidation, where decisions depend on physical observations. Existing evaluations primarily focus on end-to-end molecular reconstruction, where correct predictions do not necessarily indicate reliance on the supplied properties. We introduce MolP2SBench, a benchmark for molecular property-to-structure (P2S) reasoning in LLMs. Each task presents a masked molecular scaffold with given properties and asks the model to select the missing structure from plausible candidates. We shuffle or replace properties to test reliance on correct property-structure pairing and responsiveness to counterfactual property changes. A channel-wise leaderboard uses Property-Grounded Recovery (PGR), giving equal weight to source and counterfactual recovery. Evaluating seven models, we find that LLMs can use physical properties for structure prediction, but performance varies substantially across property types. Higher accuracy with a property does not necessarily imply greater responsiveness to it: a model may benefit more from 1H NMR than from 13C NMR, yet be less likely to change its prediction when the 1H data is replaced. LLM performance also need not reflect the relative difficulty of these properties for human chemists. Although UV-Vis spectra provide less direct structural information than NMR spectra, Astra achieves higher accuracy with UV-Vis data than with NMR peaks. MolP2SBench supports studying how LLMs extract and use physical-property information, revealing patterns that may differ from conventional chemical intuition and providing a foundation for more reliably property-grounded models for molecular and quantum-chemical inference. Code and dataset: https://github.com/DUAL-Xiao/MolP2SBench.
Related
- Prediction of Molecular Physicochemical Properties Using Molecular Structure and Characteristics
- Structure–Function Relationships of a High-Molecular-Weight Exopolysaccharide from Euhalothece sp. with Bioactive and Emulsifying Properties
- Dataset of magnetic properties, structure and synthesis of doped barium and strontium M-type hexaferrites
- Electronic Structure, Bonding Topology, and Photophysical Properties of the Low-Valent Beryllium Cluster CpBe2NO: A Multireference and Real-Space Wavefunction Analysis
- Effect of Glucose and Methyl Glucoside on the Structure and Physicochemical Properties of Liposomes
BibTeX
@misc{xie2026molp2sbench,
title = {{MolP2SBench: Do LLMs Really Reason from Molecular Properties to Structure?}},
author = {Yuhang Xie and Zhiyi Shi and Zeyi Zhang and Zhisheng Qi and Yushun Dong and Yu Wang and Xiongye Xiao},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.12522},
eprint = {2610.12522},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.12522},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - MolP2SBench: Do LLMs Really Reason from Molecular Properties to Structure?
AU - Yuhang Xie
AU - Zhiyi Shi
AU - Zeyi Zhang
AU - Zhisheng Qi
AU - Yushun Dong
AU - Yu Wang
AU - Xiongye Xiao
PY - 2026
DA - 2026/10/08
PB - arXiv
DO - 10.48550/arxiv.2610.12522
UR - https://arxiv.org/abs/2610.12522
AB - Large language models (LLMs) are increasingly used in molecular discovery and structure elucidation, where decisions depend on physical observations. Existing evaluations primarily focus on end-to-end molecular reconstruction, where correct predictions do not necessarily indicate reliance on the supplied properties. We introduce MolP2SBench, a benchmark for molecular property-to-structure (P2S) reasoning in LLMs. Each task presents a masked molecular scaffold with given properties and asks the model to select the missing structure from plausible candidates. We shuffle or replace properties to test reliance on correct property-structure pairing and responsiveness to counterfactual property changes. A channel-wise leaderboard uses Property-Grounded Recovery (PGR), giving equal weight to source and counterfactual recovery. Evaluating seven models, we find that LLMs can use physical properties for structure prediction, but performance varies substantially across property types. Higher accuracy with a property does not necessarily imply greater responsiveness to it: a model may benefit more from ^1H NMR than from ^13C NMR, yet be less likely to change its prediction when the ^1H data is replaced. LLM performance also need not reflect the relative difficulty of these properties for human chemists. Although UV-Vis spectra provide less direct structural information than NMR spectra, Astra achieves higher accuracy with UV-Vis data than with NMR peaks. MolP2SBench supports studying how LLMs extract and use physical-property information, revealing patterns that may differ from conventional chemical intuition and providing a foundation for more reliably property-grounded models for molecular and quantum-chemical inference. Code and dataset: https://github.com/DUAL-Xiao/MolP2SBench.
ER -