Agentic BAIM-LLM Evaluation (ABLE): Benchmarking LLM Use of Protein Design Tools
Abstract
We introduce ABLE, a benchmark for evaluating LLM agents' ability to use biological AI models (BAIMs), such as ProteinMPNN and AlphaFold3, in dual-use protein design workflows. ABLE assesses agent performance through a set of tasks spanning structure retrieval, sequence generation, and design validation. We evaluate 15 frontier models and find that seven refuse all tasks, while the remaining models exhibit substantial performance differences. Claude Sonnet 4 and Gemini 3 Pro achieve the highest scores across information retrieval, tool selection, and tool use. We further compare model performance on a subset of tasks against an expert human baseline. Our results suggest that current LLMs can substantially lower barriers to protein design, but remain inconsistent in planning, strategy generation, and integrating biological knowledge with tool use.
Related
- LPT: Orchestrating bounded agentic loops for automated protein binder design
- Protify: A Benchmarking Platform Revealing Task Dependence and Homology Leakage in Protein Language Model Evaluation
- De Novo Design and AlphaFold3 Evaluation of Protein Binders Targeting Specific Sites of MAP4K4
- Computational Design and Evaluation of De Novo Protein Minibinders Targeting Specific Sites of STAT3
- VibeProteinBench: An Evaluation Benchmark for Language-interfaced Vibe Protein Design
BibTeX
@misc{cai2026agentic,
title = {{Agentic BAIM-LLM Evaluation (ABLE): Benchmarking LLM Use of Protein Design Tools}},
author = {Bryce Cai and Geetha Jeyapragasan and Samira Nedungadi and Jake Yukich and Seth Donoughe},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.05818},
eprint = {2609.05818},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.05818},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Agentic BAIM-LLM Evaluation (ABLE): Benchmarking LLM Use of Protein Design Tools
AU - Bryce Cai
AU - Geetha Jeyapragasan
AU - Samira Nedungadi
AU - Jake Yukich
AU - Seth Donoughe
PY - 2026
DA - 2026/09/05
PB - arXiv
DO - 10.48550/arxiv.2609.05818
UR - https://arxiv.org/abs/2609.05818
AB - We introduce ABLE, a benchmark for evaluating LLM agents' ability to use biological AI models (BAIMs), such as ProteinMPNN and AlphaFold3, in dual-use protein design workflows. ABLE assesses agent performance through a set of tasks spanning structure retrieval, sequence generation, and design validation. We evaluate 15 frontier models and find that seven refuse all tasks, while the remaining models exhibit substantial performance differences. Claude Sonnet 4 and Gemini 3 Pro achieve the highest scores across information retrieval, tool selection, and tool use. We further compare model performance on a subset of tasks against an expert human baseline. Our results suggest that current LLMs can substantially lower barriers to protein design, but remain inconsistent in planning, strategy generation, and integrating biological knowledge with tool use.
ER -