Separating perception from reasoning in vision-language models: a model-free render ceiling for crystal structures
Abstract
Multimodal evaluations cannot say whether a vision-language model misread an image or misreasoned about it, because every existing method for separating the two places a second model in the loop. We introduce the render ceiling, a model-free reference for benchmarks built by rendering known objects: inverting the frozen cameras and re-solving cross-view correspondence recovers exactly the answer the images support. We prove the ceiling fails only through an enumerable set of projection coincidences and certify that set empty on 2,160 rendered crystal structures, so every point of a model's deficit belongs to the model. Across fourteen vision-language models, supplying exact geometry as text lifts every model yet closes under half the gap for thirteen, while a supervised vision model with no language component reads the same images at 0.8952, above every vision-language model. The instrument exposes extraction-stage fabrication that downstream accuracy would misattribute to reasoning, yields camera-placement rules for benchmark builders, and transfers to any benchmark with an invertible forward rendering.
Related
- Surfactant-Toolkit and SurfBench-200: an open computation toolkit and a benchmark of tool-augmented language-model reasoning in surfactant science
- How Chemical Evidence Shapes Mechanistic Reasoning and Prediction by Large Language Models: Fine-Grained Inference in Nucleoside-Derived Supramolecular Gelation
- Human Language Shapes the Emerging Physical Reasoning in Large Language Models
- Hierarchical Multi-agent Large Language Model Reasoning for Autonomous Heterogeneous Catalyst Discovery
- From Answers to States: Verifiable Process-Level Evaluation of Chemical Reasoning in Large Language Models
BibTeX
@misc{polat2026separating,
title = {{Separating perception from reasoning in vision-language models: a model-free render ceiling for crystal structures}},
author = {Can Polat and Mustafa Kurban and Erchin Serpedin and Hasan Kurban},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.00663},
eprint = {2609.00663},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.00663},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Separating perception from reasoning in vision-language models: a model-free render ceiling for crystal structures
AU - Can Polat
AU - Mustafa Kurban
AU - Erchin Serpedin
AU - Hasan Kurban
PY - 2026
DA - 2026/09/01
PB - arXiv
DO - 10.48550/arxiv.2609.00663
UR - https://arxiv.org/abs/2609.00663
AB - Multimodal evaluations cannot say whether a vision-language model misread an image or misreasoned about it, because every existing method for separating the two places a second model in the loop. We introduce the render ceiling, a model-free reference for benchmarks built by rendering known objects: inverting the frozen cameras and re-solving cross-view correspondence recovers exactly the answer the images support. We prove the ceiling fails only through an enumerable set of projection coincidences and certify that set empty on 2,160 rendered crystal structures, so every point of a model's deficit belongs to the model. Across fourteen vision-language models, supplying exact geometry as text lifts every model yet closes under half the gap for thirteen, while a supervised vision model with no language component reads the same images at 0.8952, above every vision-language model. The instrument exposes extraction-stage fabrication that downstream accuracy would misattribute to reasoning, yields camera-placement rules for benchmark builders, and transfers to any benchmark with an invertible forward rendering.
ER -