Physics as the label for measuring and correcting materials reasoning in multimodal models
Abstract
Vision-language and language models increasingly interpret materials data, yet benchmarks report that they hallucinate invalid properties and violate physical law. Evaluation matches final answers to scarce human labels, while discovery agents verify final proposals or density functional theory (DFT) execution. Neither measures the physical consistency of a model's reasoning chain. Materials data carries its own physics, making a large class of materials reasoning verifiable without annotation. We introduce MatPCR, a label-free benchmark whose programmatic oracles check diffraction geometry through Bragg's law, scale bars, spectral peaks, and Materials Project-grounded checks of near-hull stability, computed band-gap class, and net magnetization. We define the Physical-Consistency Rate over image and structure inputs; introduce Constraint-Grounded Self-Verification, an agentic loop whose gain survives self-refinement and equal-compute re-prompting controls; release an open verifier useful in distribution but near chance on all six held-out constraint types; and derive an exact identity for how oracle error displaces the reported rate.
Related
- Scaling LLM Agents for Materials Design through Hierarchical Collective Reasoning
- Measuring trainable degrees of freedom in materials graph neural networks: a random-subspace intrinsic dimension analysis
- Identification of constitutive materials of bi-layer soft tissues from multimodal indentations
- Hypothesis-Driven Autonomous Materials Synthesis with Multimodal LLM Agents
- Physics-Informed and Knowledge-Driven Generative AI for Autonomous Discovery of Porous Oxide Energy Materials: Opportunities and Challenges
BibTeX
@misc{kurban2026physics,
title = {{Physics as the label for measuring and correcting materials reasoning in multimodal models}},
author = {Hasan Kurban and Rasul Khanbayov and Mustafa Kurban},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.12181},
eprint = {2609.12181},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.12181},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Physics as the label for measuring and correcting materials reasoning in multimodal models
AU - Hasan Kurban
AU - Rasul Khanbayov
AU - Mustafa Kurban
PY - 2026
DA - 2026/09/10
PB - arXiv
DO - 10.48550/arxiv.2609.12181
UR - https://arxiv.org/abs/2609.12181
AB - Vision-language and language models increasingly interpret materials data, yet benchmarks report that they hallucinate invalid properties and violate physical law. Evaluation matches final answers to scarce human labels, while discovery agents verify final proposals or density functional theory (DFT) execution. Neither measures the physical consistency of a model's reasoning chain. Materials data carries its own physics, making a large class of materials reasoning verifiable without annotation. We introduce MatPCR, a label-free benchmark whose programmatic oracles check diffraction geometry through Bragg's law, scale bars, spectral peaks, and Materials Project-grounded checks of near-hull stability, computed band-gap class, and net magnetization. We define the Physical-Consistency Rate over image and structure inputs; introduce Constraint-Grounded Self-Verification, an agentic loop whose gain survives self-refinement and equal-compute re-prompting controls; release an open verifier useful in distribution but near chance on all six held-out constraint types; and derive an exact identity for how oracle error displaces the reported rate.
ER -