A protein language model for domain, functional site, and variant effect prediction
- 1Children's Hospital of Philadelphia
Abstract
Protein sequence databases are growing far faster than experimentally curated annotations, limiting functional interpretation of newly identified proteins and coding variants. We developed PALM, a protein annotation language model that jointly predicts domain and functional-site occurrence, while preserving overlapping and nested annotations through a soft-target objective. PALM integrates ESM2 with low-rank adaptation (LoRA), learned layer aggregation, and a bidirectional long short-term memory (BiLSTM) encoder. PALM was trained on ~15 million InterPro-annotated UniRef50 sequences spanning 17,951 domain and 1,383 site classes. Across 2.3 million held-out proteins, PALM achieved domain- and site-occurrence F1 scores of 0.941 and 0.877, respectively, and retained substantial performance in proteins with <20% sequence identity to the training and validation data. PALM also substantially outperformed ESM3 annotation baseline. Without training on variant-pathogenicity labels, PALM embedding disruption distinguished 151,823 ClinVar missense variants more accurately than ESM2 embedding disruption (AUROC 0.835 versus 0.710), and combining PALM embedding disruption with the ESM2 likelihood-ratio signal reached an AUROC of 0.877. The same combined zero-shot score outperformed AlphaMissense on a benchmark dataset on 128,161 missense variants, that were derived from literature but not present in ClinVar database. At the protein level, PALM embeddings showed stronger CATH-Gene3D superfamily organization than frozen ESM2 and ESM3 embeddings, and improved prediction of experimental activation and repression effects when combined with ESM2. In a downstream case study, we demonstrated the ability to use PALM-derived embedding as proxy of deep mutational scanning (DMS), showcasing the broad application of PALM. Together, PALM converts curated residue-level knowledge into transferable and mechanistically interpretable representations that connect protein sequence, function, and variant consequence.
Related
- Genetic Testing Unveils a Novel Thrombospondin‐1 Domain Containing Protein 1 Gene Variant as the Cause of Chronic Edema in a 79‐Year‐Old Woman
- Engineering a hemoglobin variant with enhanced stability and oxygen affinity using a protein language model.
- Limitations of general protein language models and public TCRpMHC data in training specificity prediction models
- Protein Language Model-Conditioned Graph Neural Networks for Multitask GPCR Ligand Activity Prediction
- Language-guided domain adaptation improves protein sequence rep-resentations for antimicrobial resistance analysis
BibTeX
@misc{wang2026protein,
title = {{A protein language model for domain, functional site, and variant effect prediction}},
author = {P. Wang and M. U. Ahsan and T. Chen and K. Wang},
howpublished = {bioRxiv},
year = {2026},
month = nov,
doi = {10.64898/2026.11.03.764008},
url = {https://www.biorxiv.org/content/10.64898/2026.11.03.764008v1},
note = {CC BY-NC},
}
RIS
TY - UNPB
TI - A protein language model for domain, functional site, and variant effect prediction
AU - P. Wang
AU - M. U. Ahsan
AU - T. Chen
AU - K. Wang
PY - 2026
DA - 2026/11/03
PB - bioRxiv
DO - 10.64898/2026.11.03.764008
UR - https://www.biorxiv.org/content/10.64898/2026.11.03.764008v1
AB - Protein sequence databases are growing far faster than experimentally curated annotations, limiting functional interpretation of newly identified proteins and coding variants. We developed PALM, a protein annotation language model that jointly predicts domain and functional-site occurrence, while preserving overlapping and nested annotations through a soft-target objective. PALM integrates ESM2 with low-rank adaptation (LoRA), learned layer aggregation, and a bidirectional long short-term memory (BiLSTM) encoder. PALM was trained on ~15 million InterPro-annotated UniRef50 sequences spanning 17,951 domain and 1,383 site classes. Across 2.3 million held-out proteins, PALM achieved domain- and site-occurrence F1 scores of 0.941 and 0.877, respectively, and retained substantial performance in proteins with <20% sequence identity to the training and validation data. PALM also substantially outperformed ESM3 annotation baseline. Without training on variant-pathogenicity labels, PALM embedding disruption distinguished 151,823 ClinVar missense variants more accurately than ESM2 embedding disruption (AUROC 0.835 versus 0.710), and combining PALM embedding disruption with the ESM2 likelihood-ratio signal reached an AUROC of 0.877. The same combined zero-shot score outperformed AlphaMissense on a benchmark dataset on 128,161 missense variants, that were derived from literature but not present in ClinVar database. At the protein level, PALM embeddings showed stronger CATH-Gene3D superfamily organization than frozen ESM2 and ESM3 embeddings, and improved prediction of experimental activation and repression effects when combined with ESM2. In a downstream case study, we demonstrated the ability to use PALM-derived embedding as proxy of deep mutational scanning (DMS), showcasing the broad application of PALM. Together, PALM converts curated residue-level knowledge into transferable and mechanistically interpretable representations that connect protein sequence, function, and variant consequence.
ER -