{
  "abstract": "MOTIVATION: Recent advances in large language models have enabled the emergence of AI scientists that aim to autonomously analyze biological data and assist scientific discovery. Despite rapid progress, it remains unclear to what extent these systems can extract meaningful biological insights from real experimental data. Existing benchmarks either evaluate reasoning in the absence of data or focus on predefined analytical outputs, failing to reflect realistic, data-driven biological research. RESULTS: Here, we introduce BAISBench (Biological AI Scientist Benchmark), a benchmark for evaluating AI scientists on real single-cell transcriptomic datasets. BAISBench comprises two tasks: cell type annotation across 15 expert-labeled datasets, and scientific discovery through 193 multiple-choice questions derived from biological conclusions reported in 41 published single-cell studies. We evaluated several representative AI scientists using BAISBench and, to provide a human performance baseline, invited five graduate-level bioinformaticians to collectively complete the same tasks. The results show that while current AI scientists fall short of fully autonomous biological discovery, they already demonstrate substantial potential in supporting data-driven biological research. These results position BAISBench as a practical benchmark for characterizing the current capabilities and limitations of AI scientists in biological research. We expect BAISBench to serve as a practical evaluation framework for guiding the development of more capable AI scientists and for helping biologists identify AI systems that can effectively support real-world research workflows. AVAILABILITY AND IMPLEMENTATION: https://github.com/EperLuo/BAISBench, https://huggingface.co/datasets/EperLuo/BaisBench.",
  "abstract_shown": true,
  "affiliations": [
    "MOE Key Laboratory of Bioinformatics and Bioinformatics Division of BNRIST, Department of Automation, Tsinghua University , Beijing, 100084,",
    "School of Software Engineering, Beijing Jiaotong University , Beijing, 100044,",
    "Department of Physiology and Pathophysiology, School of Basic Medical Sciences, Capital Medical University , Beijing, 100069,",
    "Center for Synthetic and Systems Biology, School of Life Sciences and School of Medicine, Tsinghua University , Beijing, 100084,"
  ],
  "article_type": null,
  "arxiv": null,
  "author_total": 9,
  "authors": [
    {
      "affs": [
        0
      ],
      "name": "Erpai Luo",
      "orcid": "0009-0009-4087-6027"
    },
    {
      "affs": [
        0
      ],
      "name": "Jinmeng Jia"
    },
    {
      "affs": [
        0
      ],
      "name": "Yifan Xiong"
    },
    {
      "affs": [
        1
      ],
      "name": "Xiangyu Li",
      "orcid": "0000-0002-6017-0701"
    },
    {
      "affs": [
        2
      ],
      "name": "Xiaobo Guo"
    },
    {
      "affs": [
        2
      ],
      "name": "Baoqi Yu",
      "orcid": "0009-0005-5150-2507"
    },
    {
      "affs": [
        0
      ],
      "name": "Minsheng Hao",
      "orcid": "0000-0001-6749-5659"
    },
    {
      "affs": [
        0
      ],
      "name": "Lei Wei",
      "orcid": "0000-0002-1546-6458"
    },
    {
      "affs": [
        3,
        0
      ],
      "name": "Xuegong Zhang",
      "orcid": "0000-0002-9684-5643"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/8gs70enbkcg4d.bib",
  "cited_by": 25,
  "doi": "10.1093/bioinformatics/btag227",
  "fulltext_url": null,
  "id": "8gs70enbkcg4d",
  "issns": [
    "1367-4803",
    "1367-4811"
  ],
  "landing_url": "https://doi.org/10.1093/bioinformatics/btag227",
  "language": "en",
  "licence": "CC-BY",
  "licence_label": "CC BY",
  "licence_source": "journal",
  "licence_url": "https://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "openalex",
      "1312|Single-cell and spatial transcriptomics"
    ]
  ],
  "openalex": "W7167635513",
  "pdf_url": null,
  "pmcid": "PMC13340177",
  "pmid": "42412809",
  "published": "2026-07-01",
  "publisher": "Oxford University Press",
  "ris": "https://research.draconitic.com/paper/8gs70enbkcg4d.ris",
  "sources": [
    {
      "first_seen": "2026-11-04T13:21:07Z",
      "native_id": "W7167635513",
      "source": "openalex",
      "url": "https://doi.org/10.1093/bioinformatics/btag227"
    }
  ],
  "subjects": [
    {
      "field": "biochemistry",
      "origin": "openalex:1312|Single-cell and spatial transcriptomics",
      "subfield": "molecular-biology"
    }
  ],
  "title": "Benchmarking AI scientists for omics data–driven biological discovery",
  "title_text": "Benchmarking AI scientists for omics data–driven biological discovery",
  "type": "peer-reviewed",
  "updated_at": "2026-11-08T13:20:41Z",
  "url": "https://research.draconitic.com/paper/8gs70enbkcg4d",
  "venue": "Bioinformatics",
  "versions": []
}