Benchmarking the AlphaGenome Variant Impact score against SpliceAI on post-release ClinVar classifications and experimental splicing assays
- 1AIstanbul Research Group, Turkey
Abstract
The AlphaGenome Variant Impact (AVI) score ranks variants by predicted impact, and its developers report large gains over other genome-wide scores for intronic ClinVar variants. We tested AVI on two kinds of ground truth the developers did not use, under an analysis plan set before scoring: ClinVar classifications made after the developers' ClinVar release (87,666 rare single-nucleotide variants outside AVI's training chromosomes), and six splicing assays whose variants no predictor chose. In intronic variants at least 3 bases from an exon (449 pathogenic or likely pathogenic, 28,188 benign or likely benign), AVI reached an area under the receiver operating characteristic curve (AUROC) of 0.957. It did not differ detectably from SpliceAI (difference -0.005, 95% interval -0.017 to 0.006) or Pangolin (-0.006, -0.018 to 0.007); both intervals exclude differences larger than about 0.02. AVI had a higher AUROC than CADD (+0.071). In deep intronic variants it trailed SpliceAI (-0.070, -0.118 to -0.026). Under an adapted ClinGen calibration, AVI gave strong evidence to 52% of pathogenic variants on the pooled test set, but its levels did not hold in every region. It gave benign evidence to 24% of benign variants, against 90% for SpliceAI. On the assays, AVI ranked splice-altering variants below raw SpliceAI (pooled difference -0.089, -0.129 to -0.049) and did not differ from masked SpliceAI (+0.003). SpliceAI2, released during the study, led AVI on both kinds of ground truth. AVI did not outperform splicing predictors on AUROC and gave little benign evidence. AlphaGenome outputs are provided for research use, not for diagnostic decisions.
Related
- Decision benchmarking reveals when protein foundation models add experimental value
- Experimental benchmarking of in silico annotations at prostate cancer risk loci and hypothesis-generating cross-cancer prioritisation of ovarian risk variants.
- Analysis of single nucleotide substitution in CYP21A2 gene using Alphagenome
- AlphaGenome informed machine learning for pathogenicity prediction of germline mutations
- Scooby-2: fine-tuning AlphaGenome for single-cell genomic track prediction
BibTeX
@misc{cavus2026benchmarking,
title = {{Benchmarking the AlphaGenome Variant Impact score against SpliceAI on post-release ClinVar classifications and experimental splicing assays}},
author = {O. A. Cavus and A. Kuskucu},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.11.758362},
url = {https://www.biorxiv.org/content/10.64898/2026.10.11.758362v1},
note = {CC BY-NC},
}
RIS
TY - UNPB
TI - Benchmarking the AlphaGenome Variant Impact score against SpliceAI on post-release ClinVar classifications and experimental splicing assays
AU - O. A. Cavus
AU - A. Kuskucu
PY - 2026
DA - 2026/10/11
PB - bioRxiv
DO - 10.64898/2026.10.11.758362
UR - https://www.biorxiv.org/content/10.64898/2026.10.11.758362v1
AB - The AlphaGenome Variant Impact (AVI) score ranks variants by predicted impact, and its developers report large gains over other genome-wide scores for intronic ClinVar variants. We tested AVI on two kinds of ground truth the developers did not use, under an analysis plan set before scoring: ClinVar classifications made after the developers' ClinVar release (87,666 rare single-nucleotide variants outside AVI's training chromosomes), and six splicing assays whose variants no predictor chose. In intronic variants at least 3 bases from an exon (449 pathogenic or likely pathogenic, 28,188 benign or likely benign), AVI reached an area under the receiver operating characteristic curve (AUROC) of 0.957. It did not differ detectably from SpliceAI (difference -0.005, 95% interval -0.017 to 0.006) or Pangolin (-0.006, -0.018 to 0.007); both intervals exclude differences larger than about 0.02. AVI had a higher AUROC than CADD (+0.071). In deep intronic variants it trailed SpliceAI (-0.070, -0.118 to -0.026). Under an adapted ClinGen calibration, AVI gave strong evidence to 52% of pathogenic variants on the pooled test set, but its levels did not hold in every region. It gave benign evidence to 24% of benign variants, against 90% for SpliceAI. On the assays, AVI ranked splice-altering variants below raw SpliceAI (pooled difference -0.089, -0.129 to -0.049) and did not differ from masked SpliceAI (+0.003). SpliceAI2, released during the study, led AVI on both kinds of ground truth. AVI did not outperform splicing predictors on AUROC and gave little benign evidence. AlphaGenome outputs are provided for research use, not for diagnostic decisions.
ER -