RNA3D-Decoy: A Large-Scale Benchmark Dataset of AI-Generated RNA Structure Models for Quality Assessment
- 1Hamilton College
Abstract
Recent advances in AI- RNA structure prediction have increased the availability of candidate models, but their accuracy remains variable. Developing reliable quality assessment (QA) methods requires datasets that represent contemporary prediction errors across diverse targets and molecular contexts. Here, we present RNA3D-Decoy, a large-scale benchmark dataset comprising 103,549 decoys for 677 experimentally determined targets, including 416 RNA-only structures,117 RNA structures with metal ions, and 144 RNA-protein complexes. Decoys were generated using five biomolecular predictors: AlphaFold 3, ESMFold2, Protenix, Chai-1, and Boltz-2.Each decoy is evaluated against its experimental reference us-ing TM-score, lDDT, and RMSD, with global and local quality annotations where applicable. DockQ scores additionally characterize RNA-protein interface quality. Target grouping incorporates both RNA and protein sequence relationships to support dataset partitioning. Analyses reveal distinct quality distributions across dataset levels and target-dependent differences among predictors, with each predictor contributing the highest-TM-score model for a subset of targets. Broad pooled score distributions also coexist with limited within-target score variation, particularly for RNA-protein complexes, emphasizing the importance of target-level characterization. By combining AI-generated decoys, standardized quality labels, and structured metadata, RNA3D-Decoy provides a resource for developing and evaluating methods for global quality estimation, local error prediction, and candidate-model ranking. The data set and accompanying code are available at: https://github.com/HamiltonChenLab/RNA3D-Decoy.
Related
- The first OpenBind release: An open experimental structure-affinity dataset and benchmark for structure-based AI
- Application of spectroscopic and multispectral imaging technologies on the assessment of ready-to-eat pineapple quality: A performance evaluation study of machine learning models generated from two commercial data analytics tools
- A 10-m National Agricultural Parcel Dataset of Ukraine Generated with an Enhanced Multi-Task Network and Sentinel-2 Imagery
- CY-Bench: a comprehensive benchmark dataset for sub-national crop yield forecasting
- Standardizing a human TBMNK cell assay across instrument platforms: a new framework for producing high-quality, AI-ready reference dataset
BibTeX
@misc{chen2026rna3ddecoy,
title = {{RNA3D-Decoy: A Large-Scale Benchmark Dataset of AI-Generated RNA Structure Models for Quality Assessment}},
author = {X. Chen and H. Yadi and R. Cao and H. Liu},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.07.757332},
url = {https://www.biorxiv.org/content/10.64898/2026.10.07.757332v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - RNA3D-Decoy: A Large-Scale Benchmark Dataset of AI-Generated RNA Structure Models for Quality Assessment
AU - X. Chen
AU - H. Yadi
AU - R. Cao
AU - H. Liu
PY - 2026
DA - 2026/10/07
PB - bioRxiv
DO - 10.64898/2026.10.07.757332
UR - https://www.biorxiv.org/content/10.64898/2026.10.07.757332v1
AB - Recent advances in AI- RNA structure prediction have increased the availability of candidate models, but their accuracy remains variable. Developing reliable quality assessment (QA) methods requires datasets that represent contemporary prediction errors across diverse targets and molecular contexts. Here, we present RNA3D-Decoy, a large-scale benchmark dataset comprising 103,549 decoys for 677 experimentally determined targets, including 416 RNA-only structures,117 RNA structures with metal ions, and 144 RNA-protein complexes. Decoys were generated using five biomolecular predictors: AlphaFold 3, ESMFold2, Protenix, Chai-1, and Boltz-2.Each decoy is evaluated against its experimental reference us-ing TM-score, lDDT, and RMSD, with global and local quality annotations where applicable. DockQ scores additionally characterize RNA-protein interface quality. Target grouping incorporates both RNA and protein sequence relationships to support dataset partitioning. Analyses reveal distinct quality distributions across dataset levels and target-dependent differences among predictors, with each predictor contributing the highest-TM-score model for a subset of targets. Broad pooled score distributions also coexist with limited within-target score variation, particularly for RNA-protein complexes, emphasizing the importance of target-level characterization. By combining AI-generated decoys, standardized quality labels, and structured metadata, RNA3D-Decoy provides a resource for developing and evaluating methods for global quality estimation, local error prediction, and candidate-model ranking. The data set and accompanying code are available at: https://github.com/HamiltonChenLab/RNA3D-Decoy.
ER -