Static Recovery Is Not Dynamic Stability: Dynamics-Aware Benchmarking of Protein Motif Scaffolding
Abstract
Protein motif scaffolding benchmarks assess whether generative models can build scaffolds around functionally important residues. Their conclusions depend on which motifs are tested and which criteria define success. Existing benchmarks typically use small hand-curated motif sets and static refolding criteria, but protein function often depends on conformational ensembles and transitions. Here, we address both benchmark scope and success criteria. We introduce BILBY-PRO, a systematically derived benchmark of structurally conserved functional motifs from PROSITE-linked experimental structures, comprising 220 cases from 174 PROSITE motif-pattern entries. Evaluating six recent scaffold-generation models reveals distinct patterns in target coverage and unique-solution yield: RFDiffusion and Proteina solve more targets, whereas ESM3 produces more unique successful scaffolds on the targets it solves. We further introduce BILBY-MD, a reference-based all-atom molecular dynamics evaluation framework. Applied to statically solved designs from the original MotifBench benchmark, it compares motif drift, scaffold drift, motif-contact preservation, and motif-dihedral free-energy landscapes against reference simulations. Of 55 statically solved model-problem pairs, 31 satisfy our MD-supported criterion and only 8 pass all checks. Compact, self-contained motifs are often MD-supported, whereas interface-like or context-dependent motifs fail more frequently. Static refolding success therefore provides useful evidence of local motif recovery but does not establish dynamical fidelity of the full scaffold. BILBY highlights three complementary axes of motif-scaffolding evaluation: target coverage, unique-solution yield, and MD-supported dynamical fidelity.
Related
- Single-cell-resolved dynamic immune remodeling following Omicron infection and recovery in vaccinated individuals
- Interlaboratory benchmarking of differential scanning fluorimetry reveals sources of variability in protein thermal stability parameters
- Dynamic Acylcarnitine Responses to Exercise and Recovery in Aging Skeletal Muscle: Evidence, Interpretation, and a Validation Framework
- Pyk2 scaffolding relieves Src autoinhibition to coordinate reciprocal kinase activation
- Multivalent Scaffolding by XIST E-Repeats Tunes RNA Binding Protein Cooperativity and Lowers the Phase Separation Threshold
BibTeX
@misc{bruyn2026static,
title = {{Static Recovery Is Not Dynamic Stability: Dynamics-Aware Benchmarking of Protein Motif Scaffolding}},
author = {Emile de Bruyn and Moritz Schäffler and Jannik Schneider and Sebastian R. Schmidt and Chanwoong Hwang and Aleena Siji and Utkarsh Upadhyay and Stefan Bauer and Alexander Schug and Stefan Kesselheim and Artur Yakimovich and Alina Bazarova},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.07723},
eprint = {2611.07723},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.07723},
}
RIS
TY - UNPB
TI - Static Recovery Is Not Dynamic Stability: Dynamics-Aware Benchmarking of Protein Motif Scaffolding
AU - Emile de Bruyn
AU - Moritz Schäffler
AU - Jannik Schneider
AU - Sebastian R. Schmidt
AU - Chanwoong Hwang
AU - Aleena Siji
AU - Utkarsh Upadhyay
AU - Stefan Bauer
AU - Alexander Schug
AU - Stefan Kesselheim
AU - Artur Yakimovich
AU - Alina Bazarova
PY - 2026
DA - 2026/11/05
PB - arXiv
DO - 10.48550/arxiv.2611.07723
UR - https://arxiv.org/abs/2611.07723
AB - Protein motif scaffolding benchmarks assess whether generative models can build scaffolds around functionally important residues. Their conclusions depend on which motifs are tested and which criteria define success. Existing benchmarks typically use small hand-curated motif sets and static refolding criteria, but protein function often depends on conformational ensembles and transitions. Here, we address both benchmark scope and success criteria. We introduce BILBY-PRO, a systematically derived benchmark of structurally conserved functional motifs from PROSITE-linked experimental structures, comprising 220 cases from 174 PROSITE motif-pattern entries. Evaluating six recent scaffold-generation models reveals distinct patterns in target coverage and unique-solution yield: RFDiffusion and Proteina solve more targets, whereas ESM3 produces more unique successful scaffolds on the targets it solves. We further introduce BILBY-MD, a reference-based all-atom molecular dynamics evaluation framework. Applied to statically solved designs from the original MotifBench benchmark, it compares motif drift, scaffold drift, motif-contact preservation, and motif-dihedral free-energy landscapes against reference simulations. Of 55 statically solved model-problem pairs, 31 satisfy our MD-supported criterion and only 8 pass all checks. Compact, self-contained motifs are often MD-supported, whereas interface-like or context-dependent motifs fail more frequently. Static refolding success therefore provides useful evidence of local motif recovery but does not establish dynamical fidelity of the full scaffold. BILBY highlights three complementary axes of motif-scaffolding evaluation: target coverage, unique-solution yield, and MD-supported dynamical fidelity.
ER -