BEND 2.0: Benchmarking the Capabilities Needed to Design DNA

Abstract

Genomic foundation models are increasingly used to propose DNA sequences and edits. But successful design requires capabilities that current benchmarks do not measure. Here we introduce BEND 2.0 (BENchmark for DNA language models), which tests how well models predict the effects of DNA changes and whether these predictions can guide sequence design. Across eight tasks built on five experimental datasets, we find that model choice and how representations are accessed both affect performance. Linear readouts fitted to frozen representations can match or exceed models' native outputs, and functional post-training substantially improves the models' capabilities. Linear readouts transfer well across experimental assays and genes, suggesting that models capture functional patterns beyond the data used to fit individual predictors. At the same time, strong prediction of average effects can conceal weaker prediction of context-specific effects, and accurate prediction of combined mutation effects does not imply accurate prediction of their interactions. In design tasks, model predictions can reduce the number of candidates needed to find a desired response, but still leave substantial room for better design choices, particularly for context-specific and multi-cell objectives. Our benchmark reveals that genomic models now display signs of general utility but have yet to saturate on performance in even simple tasks.

Agentic BAIM-LLM Evaluation (ABLE): Benchmarking LLM Use of Protein Design Tools
Bryce Cai, Geetha Jeyapragasan, Samira Nedungadi +2
arXiv5 Sep 2026PreprintCC BY 4.0Full text1 citedBioinformatics & computational biologyarXiv
Benchmarking interface confidence as a selection criterion for computational nanobody design against survivin's BIR-domain groove
C. Gately, X. Guan, X. Ding
bioRxiv4 Oct 2026PreprintCC BYFull textBioinformatics & computational biologybioRxiv
RegDesign-Bench: Benchmarking computational methods for cell-type-specific regulatory DNA design
J. Xu, Z. Guo, C. Zhong +6
bioRxiv18 Oct 2026PreprintCC BY-NC-NDFull textGenetics & genomicsbioRxiv
Pial arteries bend with every heartbeat
M. Viuf Skott, L. Ostergaard, D. D. Postnov
bioRxiv30 Sep 2026PreprintCC BYFull textNeurosciencebioRxiv
Circadian Derived Features for Early Discrimination Across Insomnia Severity Levels: At Least 8 Weeks of Monitoring Are Needed for Clinically Meaningful Assessment
Soheil Saghafi, Thomas C. Neylan, Qiao Li +2
arXiv23 Sep 2026PreprintCC BY-NC-SA 4.0Full textBioinformatics & computational biologyarXiv

Cite

BibTeX RIS

BibTeX

@misc{bjerregaard2026bend,
  title = {{BEND 2.0: Benchmarking the Capabilities Needed to Design DNA}},
  author = {Andreas Bjerregaard and Rachael DeVries and Dionysia Danai Brilli and Panagiotis Antoniadis and Yan Li and Anders Krogh and Wouter Boomsma and Ole Winther and Frederikke Isa Marin},
  howpublished = {arXiv},
  year = {2026},
  month = nov,
  doi = {10.48550/arxiv.2611.03806},
  eprint = {2611.03806},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.03806},
  note = {CC BY 4.0},
}

RIS

TY  - UNPB
TI  - BEND 2.0: Benchmarking the Capabilities Needed to Design DNA
AU  - Andreas Bjerregaard
AU  - Rachael DeVries
AU  - Dionysia Danai Brilli
AU  - Panagiotis Antoniadis
AU  - Yan Li
AU  - Anders Krogh
AU  - Wouter Boomsma
AU  - Ole Winther
AU  - Frederikke Isa Marin
PY  - 2026
DA  - 2026/11/02
PB  - arXiv
DO  - 10.48550/arxiv.2611.03806
UR  - https://arxiv.org/abs/2611.03806
AB  - Genomic foundation models are increasingly used to propose DNA sequences and edits. But successful design requires capabilities that current benchmarks do not measure. Here we introduce BEND 2.0 (BENchmark for DNA language models), which tests how well models predict the effects of DNA changes and whether these predictions can guide sequence design. Across eight tasks built on five experimental datasets, we find that model choice and how representations are accessed both affect performance. Linear readouts fitted to frozen representations can match or exceed models' native outputs, and functional post-training substantially improves the models' capabilities. Linear readouts transfer well across experimental assays and genes, suggesting that models capture functional patterns beyond the data used to fit individual predictors. At the same time, strong prediction of average effects can conceal weaker prediction of context-specific effects, and accurate prediction of combined mutation effects does not imply accurate prediction of their interactions. In design tasks, model predictions can reduce the number of candidates needed to find a desired response, but still leave substantial room for better design choices, particularly for context-specific and multi-cell objectives. Our benchmark reveals that genomic models now display signs of general utility but have yet to saturate on performance in even simple tasks.
ER  -

Abstract

Related

Cite