LLM-assisted analytical review catches statistical errors that human researchers miss: documented evidence from two computational biology case studies
- 1University of Toledo
Abstract
Large language models (LLMs) are increasingly used to assist with code generation and data analysis in computational biology, but their capacity to serve as analytical reviewers -- detecting statistical and methodological errors in completed analyses -- is less documented. We present two case studies in which LLM-assisted review identified consequential errors in computational biology analyses that the human researchers had not detected. In single-cell transcriptomics, review of a differential expression analysis of 22,048 cells from human striatum identified a sex composition confound (Chi-squared = 192.7, p = 8.1 x 10-44) that had produced spurious Y-chromosome gene hits; correction revealed KIF3A as the sole genuine finding (adjusted p = 1.37 x 10-14). In HRV-based stress classification, review identified subject leakage in a train-test split that allowed the same individuals in training and test sets; correction via leave-one-subject-out cross-validation produced a more reliable and higher AUC estimate (0.946 vs. 0.848). Both errors would have survived conventional peer review in the absence of systematic checking. Both corrected analyses have been submitted for peer review. These cases suggest that LLM-assisted analytical review -- using LLMs to interrogate completed analyses for confounds, leakage, and statistical artifacts -- is a practical quality assurance step that complements rather than replaces human scientific judgment.
Related
- Common attentional mechanisms, but distinct computational dynamics, across Statistical and Reward Learning
- From Wandering Thoughts to Mental Stillness: A Minimal Computational Statistical-Mechanical Model of Driven Relaxation
- Highlighting the strengths of new wearable fNIRS, precision accuracy microgenetic research designs, and new NeuroDOT processing algorithm to measure change over time in clinical populations for Speech-Language and Hearing Researchers
- Researchers as predators: are field biologists altering traits of the populations that they study?
- Experiences and Gender Demographics of Researchers in Long‐Term Individual‐Based Study Systems
BibTeX
@misc{chhaya2026llmassisted,
title = {{LLM-assisted analytical review catches statistical errors that human researchers miss: documented evidence from two computational biology case studies}},
author = {O. Chhaya and V. Ananth},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.15.759333},
url = {https://www.biorxiv.org/content/10.64898/2026.10.15.759333v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - LLM-assisted analytical review catches statistical errors that human researchers miss: documented evidence from two computational biology case studies
AU - O. Chhaya
AU - V. Ananth
PY - 2026
DA - 2026/10/15
PB - bioRxiv
DO - 10.64898/2026.10.15.759333
UR - https://www.biorxiv.org/content/10.64898/2026.10.15.759333v1
AB - Large language models (LLMs) are increasingly used to assist with code generation and data analysis in computational biology, but their capacity to serve as analytical reviewers -- detecting statistical and methodological errors in completed analyses -- is less documented. We present two case studies in which LLM-assisted review identified consequential errors in computational biology analyses that the human researchers had not detected. In single-cell transcriptomics, review of a differential expression analysis of 22,048 cells from human striatum identified a sex composition confound (Chi-squared = 192.7, p = 8.1 x 10-44) that had produced spurious Y-chromosome gene hits; correction revealed KIF3A as the sole genuine finding (adjusted p = 1.37 x 10-14). In HRV-based stress classification, review identified subject leakage in a train-test split that allowed the same individuals in training and test sets; correction via leave-one-subject-out cross-validation produced a more reliable and higher AUC estimate (0.946 vs. 0.848). Both errors would have survived conventional peer review in the absence of systematic checking. Both corrected analyses have been submitted for peer review. These cases suggest that LLM-assisted analytical review -- using LLMs to interrogate completed analyses for confounds, leakage, and statistical artifacts -- is a practical quality assurance step that complements rather than replaces human scientific judgment.
ER -