Homology-aware cross-validation strategies for generalization assessment in RNA secondary structure prediction
- 1sinc(i), UNL-CONICET
Abstract
RNA secondary structure prediction is a fundamental challenge in bioinformatics, essential for understandingthe functional roles of non-coding RNAs. Recently, deep learning models have transformed the field with impressive results, leading to critical discussions regarding the validity of current crossvalidation strategies. On the one hand, traditional random partitioning yields overoptimistic results due to data leakage from uncontrolled homology. On the other hand, removing every training sequence that even slightly resembles the testing set penalizes learning-based methods, requiring generalization to completely out-of-distribution sequences. While removing sequences and retraining a machine-learned model is straightforward, the experimental data used for development and parameter tuning of classical thermodynamic methods cannot be removed, leaving these methods to benefit from knowledge leakage. Here we critically analyze the cross-validation strategies currently used for RNA secondary structure prediction: random splitting, clustering-based splitting, and leaving one RNA family out for testing. We demonstrate that this last strategy can leave structural patterns present only in the test partition, making the learning task impossible. Overall, we find that the apparent superiority of either type of method is an artifact of the splitting strategy. We propose here two homology-aware strategies: human-learned folds, which exclude from testing the structures already captured by decades of human learning, and similarity folds, which measure performance across the full spectrum of test-totrain similarity. Both enable fair comparisons, applying the same rigorous assessment to classical and learning-based methods.
Related
- Leakage-controlled benchmarking reveals generalization limits of deep learning for protein-ligand binding affinity prediction
- Rigorous nested cross-validation and external generalization-gap analysis of a machine-learning classifier for U-251 glioblastoma cytotoxicity
- Regulon-informed cellular representations reveal task-dependent generalization in drug combination prediction
- Hemodynamic-Structural Multi-Fidelity Assessment of Endovascular Flow Diversion Strategies for Cerebral Aneurysm Treatment: An Integrated Computational and Experimental Framework
- Markov models of SHAPE data improve secondary structure prediction
BibTeX
@misc{bugnon2026homologyaware,
title = {{Homology-aware cross-validation strategies for generalization assessment in RNA secondary structure prediction}},
author = {L. Bugnon and G. Kulemeyer and M. Gerard and L. Di Persia and G. Stegmayer and D. H. Milone},
howpublished = {bioRxiv},
year = {2026},
month = jun,
doi = {10.64898/2026.06.28.735057},
url = {https://www.biorxiv.org/content/10.64898/2026.06.28.735057v2},
note = {CC BY},
}
RIS
TY - UNPB
TI - Homology-aware cross-validation strategies for generalization assessment in RNA secondary structure prediction
AU - L. Bugnon
AU - G. Kulemeyer
AU - M. Gerard
AU - L. Di Persia
AU - G. Stegmayer
AU - D. H. Milone
PY - 2026
DA - 2026/06/28
PB - bioRxiv
DO - 10.64898/2026.06.28.735057
UR - https://www.biorxiv.org/content/10.64898/2026.06.28.735057v2
AB - RNA secondary structure prediction is a fundamental challenge in bioinformatics, essential for understandingthe functional roles of non-coding RNAs. Recently, deep learning models have transformed the field with impressive results, leading to critical discussions regarding the validity of current crossvalidation strategies. On the one hand, traditional random partitioning yields overoptimistic results due to data leakage from uncontrolled homology. On the other hand, removing every training sequence that even slightly resembles the testing set penalizes learning-based methods, requiring generalization to completely out-of-distribution sequences. While removing sequences and retraining a machine-learned model is straightforward, the experimental data used for development and parameter tuning of classical thermodynamic methods cannot be removed, leaving these methods to benefit from knowledge leakage. Here we critically analyze the cross-validation strategies currently used for RNA secondary structure prediction: random splitting, clustering-based splitting, and leaving one RNA family out for testing. We demonstrate that this last strategy can leave structural patterns present only in the test partition, making the learning task impossible. Overall, we find that the apparent superiority of either type of method is an artifact of the splitting strategy. We propose here two homology-aware strategies: human-learned folds, which exclude from testing the structures already captured by decades of human learning, and similarity folds, which measure performance across the full spectrum of test-totrain similarity. Both enable fair comparisons, applying the same rigorous assessment to classical and learning-based methods.
ER -