In Other Words: Learning When to Trust Generated Protein Function Descriptions
- 1Emory University
Abstract
Generative protein-to-text models write function descriptions in free-form English but give no indication of which to believe. Indeed, free text is hard to evaluate. We introduce a large language model judge of equivalence of predicted vs. ''true'' function descriptions, validated against blinded expert curation. We also present ProtTale, our sequence-to-text model, that learns from the judge's verdicts to estimate its own reliability, pJay. On unseen Swiss-Prot enzymes, ProtTale and three other leading methods tie on accuracy but are right about different proteins. Here, ProtTale's unique correct predictions outnumber all other methods' combined. Scored from sequence alone, pJay is a plug-in filter, raising all models' performance. On short proteins, it raises precision for every tested method nearly twofold. We argue that generative annotation becomes useful not by being right more often, but by knowing when it can be trusted---a standard that every generative model in biology should meet.
Related
- In vitro -generated murine monocyte-derived dendritic cells habor a unique identity compared to other mononuclear phagocytes, largely conserved with their human and putative in vivo counterparts
- Structural basis for inhibition of the lipid flippase MurJ by phage lysis protein SglPP7 suggests convergent evolution with other Sgl peptides
- Ire1-Mediated Unfolded Protein Response and Other Biological Phenomena in Ascomycota Yeasts and Filamentous Fungi
- Application of spectroscopic and multispectral imaging technologies on the assessment of ready-to-eat pineapple quality: A performance evaluation study of machine learning models generated from two commercial data analytics tools
- Artificial intelligence-generated and algorithmically personalized music interventions and their effects on brain function, behavior, and health: a critical review
BibTeX
@misc{li2026other,
title = {{In Other Words: Learning When to Trust Generated Protein Function Descriptions}},
author = {T. Li and P. Ram and R. Tawfiq and F. Guintu and C. Yang and Y. Bromberg},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.15.757758},
url = {https://www.biorxiv.org/content/10.64898/2026.10.15.757758v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - In Other Words: Learning When to Trust Generated Protein Function Descriptions
AU - T. Li
AU - P. Ram
AU - R. Tawfiq
AU - F. Guintu
AU - C. Yang
AU - Y. Bromberg
PY - 2026
DA - 2026/10/15
PB - bioRxiv
DO - 10.64898/2026.10.15.757758
UR - https://www.biorxiv.org/content/10.64898/2026.10.15.757758v1
AB - Generative protein-to-text models write function descriptions in free-form English but give no indication of which to believe. Indeed, free text is hard to evaluate. We introduce a large language model judge of equivalence of predicted vs. ''true'' function descriptions, validated against blinded expert curation. We also present ProtTale, our sequence-to-text model, that learns from the judge's verdicts to estimate its own reliability, pJay. On unseen Swiss-Prot enzymes, ProtTale and three other leading methods tie on accuracy but are right about different proteins. Here, ProtTale's unique correct predictions outnumber all other methods' combined. Scored from sequence alone, pJay is a plug-in filter, raising all models' performance. On short proteins, it raises precision for every tested method nearly twofold. We argue that generative annotation becomes useful not by being right more often, but by knowing when it can be trusted---a standard that every generative model in biology should meet.
ER -