ct-validation: an open framework for benchmarking target-indication evidence against clinical trial outcomes
- 1GERO PTE. LTD. Translational AI and Discovery, , 133 Cecil Street 14-01 Keck Seng Tower , 069535,
Abstract
Summary: Human genetic evidence provides a strong foundation for selecting drug targets. However, consistent benchmarking of target-indication evidence against clinical trial outcomes has been hindered by proprietary data and customized pipelines. To address this, we present the Python package ct-validation, which evaluates whether target-indication pairs supported by specific evidence sources are enriched for clinical progression. This package computes relative success (RS) and odds ratios with confidence intervals across clinical phase transitions. It also supports ontology-based semantic disease matching and offers interfaces in Python, on the command line, and via the Model Context Protocol. By remapping the benchmark from Minikel et al. (2024) to the Experimental Factor Ontology, we reproduced their per-transition enrichment, with a maximum difference of only 1.4% across any transition. In a systematic 3 × 3 benchmark that independently varies the genetic evidence and clinical trial sources, we found that a single curated open clinical trial source reproduces the signal of the proprietary comparator (RS = 2.92 versus 2.65; 2.34 versus 2.55 without oncology). However, aggregating open clinical trial registries compresses this signal (RS = 1.52); this loss is primarily driven by the breadth of the indication data rather than the genetic evidence.
Availability and implementation: ct-validation is implemented in Python (version 3.11 or higher), distributed via pip, and freely available at https://github.com/gero-science/ct-validation, with support for Python, command-line, and Model Context Protocol interfaces. Archived code: https://doi.org/10.5281/zenodo.22901497. Benchmark data: https://doi.org/10.5281/zenodo.22901451
Related
- A mechanism-guided simulation framework for synthetic tax-risk data: Constraint validation and reproducible benchmarking
- A representation-level wavelet–EMD fusion framework for spectral analysis and detection of epileptiform patterns in pediatric EEG: computational validation with a roadmap for external benchmarking
- Evidence boundaries and integrated disease management of microbial disease risks in protected Cymbidium faberi production: a translational framework and validation agenda
- In Vivo Transcriptomics-Based Framework for Systematic Indication Expansion
- Evidence-to-Recommendation Framework for 2026 ACE Clinical Guideline on allergic rhinitis — diagnosis and management
BibTeX
@article{kostiuk2026ctvalidation,
title = {{ct-validation: an open framework for benchmarking target-indication evidence against clinical trial outcomes}},
author = {Klim Kostiuk and Daniel Igumnov and Peter Fedichev and Amir Feizi},
journal = {Bioinformatics},
year = {2026},
month = oct,
doi = {10.1093/bioinformatics/btag788},
url = {https://academic.oup.com/bioinformatics/article/doi/10.1093/bioinformatics/btag788/8944064},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - ct-validation: an open framework for benchmarking target-indication evidence against clinical trial outcomes
AU - Klim Kostiuk
AU - Daniel Igumnov
AU - Peter Fedichev
AU - Amir Feizi
PY - 2026
DA - 2026/10/20
JO - Bioinformatics
DO - 10.1093/bioinformatics/btag788
UR - https://academic.oup.com/bioinformatics/article/doi/10.1093/bioinformatics/btag788/8944064
AB - Summary: Human genetic evidence provides a strong foundation for selecting drug targets. However, consistent benchmarking of target-indication evidence against clinical trial outcomes has been hindered by proprietary data and customized pipelines. To address this, we present the Python package ct-validation, which evaluates whether target-indication pairs supported by specific evidence sources are enriched for clinical progression. This package computes relative success (RS) and odds ratios with confidence intervals across clinical phase transitions. It also supports ontology-based semantic disease matching and offers interfaces in Python, on the command line, and via the Model Context Protocol. By remapping the benchmark from Minikel et al. (2024) to the Experimental Factor Ontology, we reproduced their per-transition enrichment, with a maximum difference of only 1.4% across any transition. In a systematic 3 × 3 benchmark that independently varies the genetic evidence and clinical trial sources, we found that a single curated open clinical trial source reproduces the signal of the proprietary comparator (RS = 2.92 versus 2.65; 2.34 versus 2.55 without oncology). However, aggregating open clinical trial registries compresses this signal (RS = 1.52); this loss is primarily driven by the breadth of the indication data rather than the genetic evidence. Availability and implementation: ct-validation is implemented in Python (version 3.11 or higher), distributed via pip, and freely available at https://github.com/gero-science/ct-validation, with support for Python, command-line, and Model Context Protocol interfaces. Archived code: https://doi.org/10.5281/zenodo.22901497. Benchmark data: https://doi.org/10.5281/zenodo.22901451
SN - 1367-4803
SN - 1367-4811
ER -