Towards Sparse Causal Features for Zero-shot Mutation Effect Prediction in a Protein Language Model
- 1University of Massachusetts
Abstract
Protein language models (pLMs) such as ESM-2 achieve strong zero-shot mutation-effect prediction, yet the internal computations supporting these predictions remain poorly understood. We introduce a sparse feature circuit framework that combines sparse autoencoders, integrated-gradients attribution, and activation patching to identify the latent features that causally mediate zero-shot mutation effect prediction in ESM-2 650M. We evaluate this framework over 67 mutations ranging from strongly deleterious to weakly deleterious in the DNAJA1 J-domain, where ESM-2 predictions agree strongly with deep mutational scanning measurements. We find that circuits selected by indirect effect recover the model's predictions more efficiently and provide more informative biological explanations than those selected by raw activation changes, showing that activation magnitude does not necessarily reflect causal importance. We find that related substitutions reuse substantial portions of their recovered circuits, ranging from 40% to 75%, and that the shared features often represent residues in three-dimensional contact with the mutation site. To our knowledge, our work provides the first causal, feature-level account of zero-shot mutation effect prediction in a pLM.
Related
- A Context-Aware 14-3-3 Binding Predictor Enabled by Data Augmentation, Multi-Scale Biological Features, and Protein Language Model Embeddings
- Probing Physical Intuition in Large Language Models through Partner-Dependent Mutation Prediction
- Ground-truth evaluation of sparse autoencoder features on spatial, single-cell and language-model embeddings
- MAXWELL: Calibrating the probabilistic outputs of protein language models to the mutation-induced stability change landscape
- Inherent limitations of zero-shot mutation prediction in protein engineering
BibTeX
@misc{mohanty2026towards,
title = {{Towards Sparse Causal Features for Zero-shot Mutation Effect Prediction in a Protein Language Model}},
author = {S. Mohanty and M. Phutela and A. G. Green},
howpublished = {bioRxiv},
year = {2026},
month = aug,
doi = {10.64898/2026.08.28.747907},
url = {https://www.biorxiv.org/content/10.64898/2026.08.28.747907v1},
note = {CC BY-NC},
}
RIS
TY - UNPB
TI - Towards Sparse Causal Features for Zero-shot Mutation Effect Prediction in a Protein Language Model
AU - S. Mohanty
AU - M. Phutela
AU - A. G. Green
PY - 2026
DA - 2026/08/28
PB - bioRxiv
DO - 10.64898/2026.08.28.747907
UR - https://www.biorxiv.org/content/10.64898/2026.08.28.747907v1
AB - Protein language models (pLMs) such as ESM-2 achieve strong zero-shot mutation-effect prediction, yet the internal computations supporting these predictions remain poorly understood. We introduce a sparse feature circuit framework that combines sparse autoencoders, integrated-gradients attribution, and activation patching to identify the latent features that causally mediate zero-shot mutation effect prediction in ESM-2 650M. We evaluate this framework over 67 mutations ranging from strongly deleterious to weakly deleterious in the DNAJA1 J-domain, where ESM-2 predictions agree strongly with deep mutational scanning measurements. We find that circuits selected by indirect effect recover the model's predictions more efficiently and provide more informative biological explanations than those selected by raw activation changes, showing that activation magnitude does not necessarily reflect causal importance. We find that related substitutions reuse substantial portions of their recovered circuits, ranging from 40% to 75%, and that the shared features often represent residues in three-dimensional contact with the mutation site. To our knowledge, our work provides the first causal, feature-level account of zero-shot mutation effect prediction in a pLM.
ER -