A systematic comparison of single-cell perturbation response prediction models
10 affiliations
- 1GMU-GIBH Joint School of Life Sciences, Guangzhou Medical University, Guangzhou, China
- 2Guangzhou National Laboratory, Guangzhou, China
- 3The University of Hong Kong, Hong Kong, China
- 4Research Center for Industries of the Future, Westlake University, Hangzhou, China
- 5School of Life Sciences, Westlake University, Hangzhou, China
- 6Westlake Institute for Advanced Study, Hangzhou, China
- 7Westlake Laboratory of Life Sciences and Biomedicine, Hangzhou, China
- 8Renaissance Era Innovation Technology Co., Ltd, China
- 9Global Institute of Future Technology, Shanghai Jiao Tong University, Shanghai, China
- 10State Key Laboratory of Respiratory Disease, National Clinical Research Center for Respiratory Disease, Guangzhou Institute of Respiratory Health, The First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
Abstract
Predicting single-cell transcriptional responses to perturbations is central to dissecting gene regulation and accelerating therapeutic design, yet the field lacks a rigorous, task-spanning assessment of model behavior. We present a large-scale benchmark of 13 representative methods and baselines across 25 datasets spanning diverse perturbation modalities and species, including two primary immune-cell drug-response resources. We evaluated three core tasks-generalization to unseen single-gene perturbations, prediction of combinatorial interactions, and transfer across cell types-using 24 metrics covering expression-level accuracy, relative changes, differential expression (DE) recovery, and distributional similarity. Across tasks, performance depended strongly on perturbation effect size and evaluation perspective: Expression-level agreement was the highest for small-effect perturbations resembling controls, whereas delta- and DE-based metrics improved with larger effects, providing clearer signals. Models shared a conservative bias, with fine-tuned foundation models compressing variance and underestimating synergistic effects in combinations. PerturbNet showed superior recovery of DE signatures in Tasks 1 and 2, while no method consistently generalized across cell types in Task 3, where biological consistency dominated outcomes. This benchmark establishes current methodological limits, clarifies that different metrics probe distinct biological signals rather than redundant summaries of the same prediction problem, and provides a foundation for developing virtual-cell models that more faithfully capture heterogeneous perturbation responses.
Related
- Dynamic graph prior-guided latent adaptation for generalizable single-cell perturbation response prediction.
- Systematic comparison of affinity plasma proteomic technologies across multiple cohorts reveals generalizability of proteomic prediction
- D-SPIN constructs regulatory network models from scRNA-seq that reveal organizing principles of perturbation response
- Latent causal diffusions for single-cell perturbation modeling.
- iGPSC-DB: a comprehensive database for an inferred single-cell genetic perturbation atlas
BibTeX
@article{li2026systematic,
title = {{A systematic comparison of single-cell perturbation response prediction models}},
author = {Lanxiang Li and Yue You and Yunlin Fu and Wenyu Liao and Xueying Fan and Shihong Lu and Ye Cao and Bo Li and Wenle Ren and Jiaming Kong and Shuangjia Zheng and Jizheng Chen and Xiaodong Liu and Luyi Tian},
journal = {Science Advances},
year = {2026},
month = sep,
doi = {10.1126/sciadv.aed3414},
url = {https://doi.org/10.1126/sciadv.aed3414},
note = {CC BY},
}
RIS
TY - JOUR
TI - A systematic comparison of single-cell perturbation response prediction models
AU - Lanxiang Li
AU - Yue You
AU - Yunlin Fu
AU - Wenyu Liao
AU - Xueying Fan
AU - Shihong Lu
AU - Ye Cao
AU - Bo Li
AU - Wenle Ren
AU - Jiaming Kong
AU - Shuangjia Zheng
AU - Jizheng Chen
AU - Xiaodong Liu
AU - Luyi Tian
PY - 2026
DA - 2026/09/09
JO - Science Advances
DO - 10.1126/sciadv.aed3414
UR - https://doi.org/10.1126/sciadv.aed3414
AB - Predicting single-cell transcriptional responses to perturbations is central to dissecting gene regulation and accelerating therapeutic design, yet the field lacks a rigorous, task-spanning assessment of model behavior. We present a large-scale benchmark of 13 representative methods and baselines across 25 datasets spanning diverse perturbation modalities and species, including two primary immune-cell drug-response resources. We evaluated three core tasks-generalization to unseen single-gene perturbations, prediction of combinatorial interactions, and transfer across cell types-using 24 metrics covering expression-level accuracy, relative changes, differential expression (DE) recovery, and distributional similarity. Across tasks, performance depended strongly on perturbation effect size and evaluation perspective: Expression-level agreement was the highest for small-effect perturbations resembling controls, whereas delta- and DE-based metrics improved with larger effects, providing clearer signals. Models shared a conservative bias, with fine-tuned foundation models compressing variance and underestimating synergistic effects in combinations. PerturbNet showed superior recovery of DE signatures in Tasks 1 and 2, while no method consistently generalized across cell types in Task 3, where biological consistency dominated outcomes. This benchmark establishes current methodological limits, clarifies that different metrics probe distinct biological signals rather than redundant summaries of the same prediction problem, and provides a foundation for developing virtual-cell models that more faithfully capture heterogeneous perturbation responses.
SN - 2375-2548
ER -