Reading and Steering Representations of Materials-Science Mechanisms in an Open-Weight Language Model
Abstract
Large language models can answer scientific questions, yet a correct output does not reveal whether the model represents or uses the governing physics. Here, using three open-weight Gemma 4 models (google/gemma-4-E4B-it, google/gemma-4-12B-it, google/gemma-4-31B-it) we identify three experimentally separable signatures of materials-science mechanism information: selective concept readability, relational encoding of qualitative constitutive orientation, and causal, context-dependent control of constrained engineering answers. We combine matched direct and Jacobian vocabulary readouts, option-free state geometry, a 60-law counterfactual benchmark and causal interventions. In 50 held-out materials descriptions, three independently fitted Jacobian lenses reproduced concept ranks, and target-free word sets from both readouts enabled blinded identification of 9 of 10 mechanism families. A separate 72-prompt benchmark produced mechanism-specific hidden-state neighborhoods, but an exact graph audit showed that this apparent physical organization was equally explained by numerical comparison. We therefore compared otherwise identical prompts in which only the direction of the physical input was reversed, asking whether the resulting hidden-state movement followed the supplied constitutive law. These state transformations ordered direct, physically neutral and inverse laws across 60 frozen relations and correctly oriented 39 of 40 directional laws, whereas lexical controls were near chance. Bidirectional interventions shifted answer probabilities toward or away from the physically appropriate outcome across all 12 matched cases, while counterfactual state patches transferred opposing decision signals across mechanisms and answer formats. Physical relationships were therefore more visible in controlled state changes than in absolute states alone.
Related
- Large Language Model Steering of Acousto-Optic Tweezer Arrays
- DiffGA: Steering Frozen Diffusion Models for Materials Design via Evolutionary Search
- MASON: Materials Atomistic Structure builder Operated in Natural language
- Automated Materials Data and Knowledge Extraction using Machine Learning and Large Language Models
- Deformation mechanisms of plasticized starch materials
BibTeX
@misc{buehler2026reading,
title = {{Reading and Steering Representations of Materials-Science Mechanisms in an Open-Weight Language Model}},
author = {Markus J. Buehler},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2607.20058},
eprint = {2607.20058},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.20058},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Reading and Steering Representations of Materials-Science Mechanisms in an Open-Weight Language Model
AU - Markus J. Buehler
PY - 2026
DA - 2026/09/03
PB - arXiv
DO - 10.48550/arxiv.2607.20058
UR - https://arxiv.org/abs/2607.20058
AB - Large language models can answer scientific questions, yet a correct output does not reveal whether the model represents or uses the governing physics. Here, using three open-weight Gemma 4 models (google/gemma-4-E4B-it, google/gemma-4-12B-it, google/gemma-4-31B-it) we identify three experimentally separable signatures of materials-science mechanism information: selective concept readability, relational encoding of qualitative constitutive orientation, and causal, context-dependent control of constrained engineering answers. We combine matched direct and Jacobian vocabulary readouts, option-free state geometry, a 60-law counterfactual benchmark and causal interventions. In 50 held-out materials descriptions, three independently fitted Jacobian lenses reproduced concept ranks, and target-free word sets from both readouts enabled blinded identification of 9 of 10 mechanism families. A separate 72-prompt benchmark produced mechanism-specific hidden-state neighborhoods, but an exact graph audit showed that this apparent physical organization was equally explained by numerical comparison. We therefore compared otherwise identical prompts in which only the direction of the physical input was reversed, asking whether the resulting hidden-state movement followed the supplied constitutive law. These state transformations ordered direct, physically neutral and inverse laws across 60 frozen relations and correctly oriented 39 of 40 directional laws, whereas lexical controls were near chance. Bidirectional interventions shifted answer probabilities toward or away from the physically appropriate outcome across all 12 matched cases, while counterfactual state patches transferred opposing decision signals across mechanisms and answer formats. Physical relationships were therefore more visible in controlled state changes than in absolute states alone.
ER -