Benchmarking short-read germline structural variant calling highlights advantages of using ensembles of tools and small impact of graph genome alignment
- 1MDxCORE, Rigshospitalet, Copenhagen University Hospital, Copenhagen, Denmark
- 2Section for Computational and RNA Biology, Department of Biology, University of Copenhagen, Copenhagen, Denmark
Abstract
Background: Structural variants (SVs) in the human genome play an important role in health and disease. Identification of SVs is commonly performed using short-read sequencing. However, because sequenced fragments are typically shorter than the variants themselves, accurate SV calling remains a challenging problem. Previous benchmarking studies consistently find substantial variability in performance among SV calling tools, with disagreements largely driven by differences in underlying algorithms.
Results: We evaluate 14 high-performing SV calling tools using four samples from a newly published, high-confidence pedigree-based truth set, together with an in-house dataset. Its recent release minimizes the likelihood that it was used for training by tools, thereby reducing the risk of over-fitting. We implemented frequency filtering that reduced the downstream variants interpretation load by nearly half. We also assess alignments from a graph genome assembly, but found only small effect on performance, compared to a linear reference. DRAGEN achieves the highest overall performance, with a mean recall of 0.36 and mean F1 score of 0.51. Among the open-source tools, Manta and Dysgu achieve the highest recall, both with a mean recall of 0.26, while Manta achieves the highest mean F1 score of 0.41. We also evaluate several multi-caller ensemble strategies, which in some settings achieved higher recall than any individual tool. Across both ensemble strategies, Dysgu, Octopus, Manta, and Tardis are part of the top-performing sets.
Conclusions: Our findings highlight the importance of designing SV calling strategies using one or more tools according to the intended application, while balancing accuracy, recall, computational resources, reference choice, and clinical interpretation burden.
Supplementary Information: The online version contains supplementary material available at https://doi.org/10.1186/s13059-026-04219-3.
Related
- An atlas of functional ensembles in the zebrafish hindbrain links structural and molecular features to behavior
- PockFlex: a web server for flexibility-aware binding site identification and prioritisation from structural ensembles
- The Genome In A Bottle HG002 assembly-based variant benchmark set enables comprehensive benchmarking of small and structural variants.
- Structural Variant benchmarking frameworks: Parameterization, matching logic, and evaluation assumptions presented through HG002 and NA12878
- Patient-derived ascites reveals functional advantages of γδCAR T-cells within the ovarian cancer-like tumor microenvironment
BibTeX
@article{guo2026benchmarking,
title = {{Benchmarking short-read germline structural variant calling highlights advantages of using ensembles of tools and small impact of graph genome alignment}},
author = {Yuliu Guo and Emilie Sofie Engdal and Miyako Kodama and Drew Kaley Ann Thompson and Jiayi Yao and Alban Laus Obel Slabowska and Mònica Aguilà-Sans and Eliana Buenaventura and Anna Reimer Hansen and Andreas Ørslev Rasmussen and Frederik Otzen Bagger},
journal = {Genome Biology},
year = {2026},
month = aug,
doi = {10.1186/s13059-026-04219-3},
url = {https://doi.org/10.1186/s13059-026-04219-3},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - JOUR
TI - Benchmarking short-read germline structural variant calling highlights advantages of using ensembles of tools and small impact of graph genome alignment
AU - Yuliu Guo
AU - Emilie Sofie Engdal
AU - Miyako Kodama
AU - Drew Kaley Ann Thompson
AU - Jiayi Yao
AU - Alban Laus Obel Slabowska
AU - Mònica Aguilà-Sans
AU - Eliana Buenaventura
AU - Anna Reimer Hansen
AU - Andreas Ørslev Rasmussen
AU - Frederik Otzen Bagger
PY - 2026
DA - 2026/08/07
JO - Genome Biology
DO - 10.1186/s13059-026-04219-3
UR - https://doi.org/10.1186/s13059-026-04219-3
AB - Background: Structural variants (SVs) in the human genome play an important role in health and disease. Identification of SVs is commonly performed using short-read sequencing. However, because sequenced fragments are typically shorter than the variants themselves, accurate SV calling remains a challenging problem. Previous benchmarking studies consistently find substantial variability in performance among SV calling tools, with disagreements largely driven by differences in underlying algorithms. Results: We evaluate 14 high-performing SV calling tools using four samples from a newly published, high-confidence pedigree-based truth set, together with an in-house dataset. Its recent release minimizes the likelihood that it was used for training by tools, thereby reducing the risk of over-fitting. We implemented frequency filtering that reduced the downstream variants interpretation load by nearly half. We also assess alignments from a graph genome assembly, but found only small effect on performance, compared to a linear reference. DRAGEN achieves the highest overall performance, with a mean recall of 0.36 and mean F1 score of 0.51. Among the open-source tools, Manta and Dysgu achieve the highest recall, both with a mean recall of 0.26, while Manta achieves the highest mean F1 score of 0.41. We also evaluate several multi-caller ensemble strategies, which in some settings achieved higher recall than any individual tool. Across both ensemble strategies, Dysgu, Octopus, Manta, and Tardis are part of the top-performing sets. Conclusions: Our findings highlight the importance of designing SV calling strategies using one or more tools according to the intended application, while balancing accuracy, recall, computational resources, reference choice, and clinical interpretation burden. Supplementary Information: The online version contains supplementary material available at https://doi.org/10.1186/s13059-026-04219-3.
SN - 1474-7596
SN - 1474-760X
ER -