Benchmarking Large Language Models for Transcriptomic Data Analysis and Visualization: Multi-Model Workflow Evaluation
- 1Sandia National Laboratories
Abstract
This study presents a comprehensive benchmarking evaluation of large language models(LLMs) and agentic coding systems for automated analysis and visualization workflows for transcriptomic data, in particular human responses to COVID-19 infection. Multiple models, including ChatGPT 5.5, Claude Sonnet 4.6, Gemini 3.1, and Llama4-Scout, were assessed across a series of scientific prompts designed to test their ability to reproduce workflow components such as dimensionality reduction, clustering, outlier detection, reproducibility, and code generation. The benchmark protocol involved varying prompt specificity and examined both quantitative metrics such as clustering scores and outlier counts as well as qualitative aspects such as code quality and reproducibility. Results indicate that prompt specificity improves code structure and methodological correctness, but quantitative performance depends heavily on preprocessing choices and execution environment. While some models produced strong code and reproducible outputs, alignment with scientific targets like label-based clustering was inconsistent. The findings highlight the promise and limitations of current LLMs for automating scientific data analysis and visualization and suggest directions for future benchmarking and workflow standardization.
Related
- Protify: A Benchmarking Platform Revealing Task Dependence and Homology Leakage in Protein Language Model Evaluation
- DBCLS BioHackathon 2026 report: Mass spectrometry data analysis workflow and visualization towards trans-omics research
- OTTER: Orchestrated Transcriptomic, Tumor-xenograft (PDX/CDX), and Epigenomic Reporting Workflow
- Benchmarking Large Language Models for Assay-Aware Drug-Target Binding Affinity Prediction
- OpenLipid: a large language model workflow for targeted analysis of DIA mass spectrometry data in lipidomics
BibTeX
@misc{iyer2026benchmarking,
title = {{Benchmarking Large Language Models for Transcriptomic Data Analysis and Visualization: Multi-Model Workflow Evaluation}},
author = {A. Iyer and A. Subedi and K. S. Butler and R. Krishnakumar},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.22.761023},
url = {https://www.biorxiv.org/content/10.64898/2026.10.22.761023v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Benchmarking Large Language Models for Transcriptomic Data Analysis and Visualization: Multi-Model Workflow Evaluation
AU - A. Iyer
AU - A. Subedi
AU - K. S. Butler
AU - R. Krishnakumar
PY - 2026
DA - 2026/10/22
PB - bioRxiv
DO - 10.64898/2026.10.22.761023
UR - https://www.biorxiv.org/content/10.64898/2026.10.22.761023v1
AB - This study presents a comprehensive benchmarking evaluation of large language models(LLMs) and agentic coding systems for automated analysis and visualization workflows for transcriptomic data, in particular human responses to COVID-19 infection. Multiple models, including ChatGPT 5.5, Claude Sonnet 4.6, Gemini 3.1, and Llama4-Scout, were assessed across a series of scientific prompts designed to test their ability to reproduce workflow components such as dimensionality reduction, clustering, outlier detection, reproducibility, and code generation. The benchmark protocol involved varying prompt specificity and examined both quantitative metrics such as clustering scores and outlier counts as well as qualitative aspects such as code quality and reproducibility. Results indicate that prompt specificity improves code structure and methodological correctness, but quantitative performance depends heavily on preprocessing choices and execution environment. While some models produced strong code and reproducible outputs, alignment with scientific targets like label-based clustering was inconsistent. The findings highlight the promise and limitations of current LLMs for automating scientific data analysis and visualization and suggest directions for future benchmarking and workflow standardization.
ER -