Probing Physical Intuition in Large Language Models through Partner-Dependent Mutation Prediction
- 1Department of Bioinformatics and Biostatistics, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University
Abstract
Structure-guided mutation design requires relating a proposed substitution to its molecular environment to anticipate its effects. Existing evaluations of large language models (LLMs) in protein engineering emphasize sequence-based prediction or tool-integrated workflows, leaving unclear whether LLMs themselves can infer mutation effects from structural evidence without dedicated predictors. We evaluate physical intuition-qualitative reasoning that connects structural observations with molecular interaction principles-using a benchmark that tests the same substitution in the same protein across different binding partners, with predictions based on local atomic coordinates and no dedicated mutation-effect predictors. In the tested comparisons, the best-performing LLM achieved group accuracy comparable to the evaluated energy predictors and Biomni, an LLM-driven tool workflow, while the methods differed in their prediction patterns. Most evaluated LLMs achieved higher group accuracy for larger affinity improvements. Structural interventions support selective dependence on mutation-site and partner information, while assessable structural claims largely agree with the supplied coordinates. These findings support direct LLM structural reasoning as a basis for developing rational mutation-design tools that connect molecular observations with partner-specific effects.
Related
- Towards Sparse Causal Features for Zero-shot Mutation Effect Prediction in a Protein Language Model
- Doublecortin mutation disrupts lattice-dependent proximity to reveal cytoskeletal networks in developing neurons
- MutCleaner: Cleaning and Standardizing Biological Mutation Datasets for Variant Effect Prediction
- Inherent limitations of zero-shot mutation prediction in protein engineering
- Mutation-resolved ABL1 inhibitor resistance prediction under mutation-disjoint and temporal cross-study validation
BibTeX
@misc{cheng2026probing,
title = {{Probing Physical Intuition in Large Language Models through Partner-Dependent Mutation Prediction}},
author = {L. Cheng and K. Zheng and W. Huang and H.-F. Chen and Z. Yu},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.09.757856},
url = {https://www.biorxiv.org/content/10.64898/2026.10.09.757856v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Probing Physical Intuition in Large Language Models through Partner-Dependent Mutation Prediction
AU - L. Cheng
AU - K. Zheng
AU - W. Huang
AU - H.-F. Chen
AU - Z. Yu
PY - 2026
DA - 2026/10/09
PB - bioRxiv
DO - 10.64898/2026.10.09.757856
UR - https://www.biorxiv.org/content/10.64898/2026.10.09.757856v1
AB - Structure-guided mutation design requires relating a proposed substitution to its molecular environment to anticipate its effects. Existing evaluations of large language models (LLMs) in protein engineering emphasize sequence-based prediction or tool-integrated workflows, leaving unclear whether LLMs themselves can infer mutation effects from structural evidence without dedicated predictors. We evaluate physical intuition-qualitative reasoning that connects structural observations with molecular interaction principles-using a benchmark that tests the same substitution in the same protein across different binding partners, with predictions based on local atomic coordinates and no dedicated mutation-effect predictors. In the tested comparisons, the best-performing LLM achieved group accuracy comparable to the evaluated energy predictors and Biomni, an LLM-driven tool workflow, while the methods differed in their prediction patterns. Most evaluated LLMs achieved higher group accuracy for larger affinity improvements. Structural interventions support selective dependence on mutation-site and partner information, while assessable structural claims largely agree with the supplied coordinates. These findings support direct LLM structural reasoning as a basis for developing rational mutation-design tools that connect molecular observations with partner-specific effects.
ER -