Variant effect predictors are shaping the ClinVar classifications used to train and evaluate them
- 1University of Edinburgh
Abstract
Variant effect predictors (VEPs) are playing an increasingly influential role in clinical variant classification. Following recent calibration efforts and the development of points-based classification standards, computational predictions can now carry enough weight to move variants out of uncertain significance with little additional evidence. While clinically valuable, this increases the potential for classification-level circularity, where VEP predictions help determine the clinical classifications later used to train, benchmark and calibrate VEPs. Here, we show that computational evidence is already leaving a detectable signature in ClinVar. Among rare missense variants from a major submitter, likely benign classifications shifted markedly towards more benign scores and show a sharp step change at the calibrated moderate-evidence threshold of BayesDel_noAF. This signal extends to nearly all 52 predictors examined, including population-free models unlikely to have been used in classification, probably reflecting shared evolutionary information. Classifications informed by computational evidence therefore cannot be treated as independent ground truth, and temporal holdouts cannot eliminate this circularity. To prevent a self-reinforcing feedback loop between prediction and classification, we recommend that ClinVar submissions routinely record the evidence criteria and computational tools supporting each classification in a standardised format, alongside greater use of classification-independent benchmarks.
Related
- Benchmarking the AlphaGenome Variant Impact score against SpliceAI on post-release ClinVar classifications and experimental splicing assays
- Why variant effect predictors and multiplexed assays agree and disagree
- Leakage-Controlled Benchmarking of Variant Effect Predictors and Development of Gradient-Boosting Consensus Model
- Combining MAVEs and computational predictors improves variant classification across ancestries in hereditary cancer genes
- Homology-Based Variant-Effect Predictors Break Down on Cytochrome P450 Pharmacogenes
BibTeX
@misc{tao2026variant,
title = {{Variant effect predictors are shaping the ClinVar classifications used to train and evaluate them}},
author = {M. Tao and J. A. Marsh},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.24.761512},
url = {https://www.biorxiv.org/content/10.64898/2026.10.24.761512v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Variant effect predictors are shaping the ClinVar classifications used to train and evaluate them
AU - M. Tao
AU - J. A. Marsh
PY - 2026
DA - 2026/10/24
PB - bioRxiv
DO - 10.64898/2026.10.24.761512
UR - https://www.biorxiv.org/content/10.64898/2026.10.24.761512v1
AB - Variant effect predictors (VEPs) are playing an increasingly influential role in clinical variant classification. Following recent calibration efforts and the development of points-based classification standards, computational predictions can now carry enough weight to move variants out of uncertain significance with little additional evidence. While clinically valuable, this increases the potential for classification-level circularity, where VEP predictions help determine the clinical classifications later used to train, benchmark and calibrate VEPs. Here, we show that computational evidence is already leaving a detectable signature in ClinVar. Among rare missense variants from a major submitter, likely benign classifications shifted markedly towards more benign scores and show a sharp step change at the calibrated moderate-evidence threshold of BayesDel_noAF. This signal extends to nearly all 52 predictors examined, including population-free models unlikely to have been used in classification, probably reflecting shared evolutionary information. Classifications informed by computational evidence therefore cannot be treated as independent ground truth, and temporal holdouts cannot eliminate this circularity. To prevent a self-reinforcing feedback loop between prediction and classification, we recommend that ClinVar submissions routinely record the evidence criteria and computational tools supporting each classification in a standardised format, alongside greater use of classification-independent benchmarks.
ER -