Continuous Integration and Software Quality in Scientific Software: A Large-Scale Empirical Analysis of GitHub Repositories
- 1University of Arizona
Abstract
Bioinformatics software plays a central role in biological research, yet many academic projects lack robust testing and continuous integration (CI) practices. This absence can compromise reproducibility, accessibility, and long-term maintainability. In this study, we conduct an empirical analysis of GitHub repositories spanning bioinformatics and other scientific software domains to examine the relationship between CI, software testing, and software quality. Repository discovery was performed using domain-specific search terms via GitHub's Application Programming Interface (API). This was followed by a layered data extraction pipeline, which included a Large Language Model (LLM) validation step, that connected repositories to published works, institutions, authors, funding, and citation counts. By integrating repository activity, community engagement, and backlog health into a composite Repository Quality Index (RQI), we find that the presence of CI and software tests is significantly associated with higher repository quality. Additionally, repositories with higher academic citation counts tend to cluster in higher RQI tiers, suggesting that software quality, which is driven by proper testing and integration, is associated with academic impact and funding status. These results demonstrate that adopting industry-standard development practices, such as CI and software testing, is a tractable path toward more sustainable and impactful academic software and funding opportunities.
Related
- hpci improves reproducibility of computational biology research by integrating Continuous Integration with High Performance Computing clusters.
- Germ granules act as repositories for RNA and protein molecules essential for zebrafish germline development
- Herbaria as repositories of past soil properties and historical soil DNA
- Integrative multi-omics re-analysis of public sequencing and metabolomics repositories defines the mucosal immune and metabolic landscape of Helicobacter pylori infection and supports CagW as a rational mRNA-LNP vaccine antigen
- Mapping bioimage-analysis software adoption in microscopy
BibTeX
@misc{mellon2026continuous,
title = {{Continuous Integration and Software Quality in Scientific Software: A Large-Scale Empirical Analysis of GitHub Repositories}},
author = {W. J. Compton Mellon and M. Sy and N. Gangrade and T. Jakobi},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.01.755028},
url = {https://www.biorxiv.org/content/10.64898/2026.10.01.755028v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Continuous Integration and Software Quality in Scientific Software: A Large-Scale Empirical Analysis of GitHub Repositories
AU - W. J. Compton Mellon
AU - M. Sy
AU - N. Gangrade
AU - T. Jakobi
PY - 2026
DA - 2026/10/01
PB - bioRxiv
DO - 10.64898/2026.10.01.755028
UR - https://www.biorxiv.org/content/10.64898/2026.10.01.755028v1
AB - Bioinformatics software plays a central role in biological research, yet many academic projects lack robust testing and continuous integration (CI) practices. This absence can compromise reproducibility, accessibility, and long-term maintainability. In this study, we conduct an empirical analysis of GitHub repositories spanning bioinformatics and other scientific software domains to examine the relationship between CI, software testing, and software quality. Repository discovery was performed using domain-specific search terms via GitHub's Application Programming Interface (API). This was followed by a layered data extraction pipeline, which included a Large Language Model (LLM) validation step, that connected repositories to published works, institutions, authors, funding, and citation counts. By integrating repository activity, community engagement, and backlog health into a composite Repository Quality Index (RQI), we find that the presence of CI and software tests is significantly associated with higher repository quality. Additionally, repositories with higher academic citation counts tend to cluster in higher RQI tiers, suggesting that software quality, which is driven by proper testing and integration, is associated with academic impact and funding status. These results demonstrate that adopting industry-standard development practices, such as CI and software testing, is a tractable path toward more sustainable and impactful academic software and funding opportunities.
ER -