State space model for new generation network alternative to transformers: a survey
- 1School of Computer Science and Technology, Anhui University
- 2School of Artificial Intelligence, Anhui University
- 3Peng Cheng Laboratory
Abstract
In the post-deep learning era, the Transformer architecture has demonstrated its powerful performance in pre-trained large models and various downstream tasks. However, the enormous computational demands of Transformers have deterred many researchers. To reduce the complexity of attention models, numerous efforts have been made to design more efficient methods. Among them, the state space model (SSM), as a possible replacement for the self-attention based Transformer model, has drawn more and more attention in recent years. This paper provides the first comprehensive review of SSM works and offers experimental comparisons and analyses to better demonstrate SSM’s features and advantages. First, we provide a detailed description of the principles to help the readers quickly grasp the key ideas of the SSM. Then, we will review existing SSMs and their various applications, including those in the fields of natural language processing, computer vision, graphs, multi-modal and multi-media, point cloud and event stream, and other domains. Next, we provide statistical comparisons and analyses of these models to help readers understand how different structures affect performance on various tasks. Finally, we suggest potential research directions that could further advance the development of the theoretical model and propose possible research points in this direction to better promote the application of SSMs.
Related
- A CD4-lineage erythro–myeloid progenitor within the adult bone marrow defines an alternative pathway for macrophage generation
- Neural oscillations and network specialisation in early childhood captured with next-generation magnetoencephalography
- Regional and network-level resting state functional connectivity markers of pharmacological treatment resistance in obsessive-compulsive disorder
- EU CAP Network Focus Group 'Innovative and sustainable ways to strengthen the role of farmers in revitalising the European wool value chain' Mini Paper 6 State of the art of the wool value chain in Europe
- Careers in Scientific Editing: Results from the 2024 Scientific Editors Network (ScENe) Survey
BibTeX
@article{wang2026state,
title = {{State space model for new generation network alternative to transformers: a survey}},
author = {Xiao Wang and Shiao Wang and Yuhe Ding and Yuehang Li and Wentao Wu and Yao Rong and Weizhe Kong and Ju Huang and Shihao Li and Haoxiang Yang and Ziwen Wang and Bo Jiang and Chenglong Li and Yaowei Wang and Yonghong Tian and Jin Tang},
journal = {Visual Intelligence},
year = {2026},
month = oct,
doi = {10.1007/s44267-026-00133-1},
url = {https://doi.org/10.1007/s44267-026-00133-1},
note = {CC BY},
}
RIS
TY - JOUR
TI - State space model for new generation network alternative to transformers: a survey
AU - Xiao Wang
AU - Shiao Wang
AU - Yuhe Ding
AU - Yuehang Li
AU - Wentao Wu
AU - Yao Rong
AU - Weizhe Kong
AU - Ju Huang
AU - Shihao Li
AU - Haoxiang Yang
AU - Ziwen Wang
AU - Bo Jiang
AU - Chenglong Li
AU - Yaowei Wang
AU - Yonghong Tian
AU - Jin Tang
PY - 2026
DA - 2026/10/25
JO - Visual Intelligence
DO - 10.1007/s44267-026-00133-1
UR - https://doi.org/10.1007/s44267-026-00133-1
AB - In the post-deep learning era, the Transformer architecture has demonstrated its powerful performance in pre-trained large models and various downstream tasks. However, the enormous computational demands of Transformers have deterred many researchers. To reduce the complexity of attention models, numerous efforts have been made to design more efficient methods. Among them, the state space model (SSM), as a possible replacement for the self-attention based Transformer model, has drawn more and more attention in recent years. This paper provides the first comprehensive review of SSM works and offers experimental comparisons and analyses to better demonstrate SSM’s features and advantages. First, we provide a detailed description of the principles to help the readers quickly grasp the key ideas of the SSM. Then, we will review existing SSMs and their various applications, including those in the fields of natural language processing, computer vision, graphs, multi-modal and multi-media, point cloud and event stream, and other domains. Next, we provide statistical comparisons and analyses of these models to help readers understand how different structures affect performance on various tasks. Finally, we suggest potential research directions that could further advance the development of the theoretical model and propose possible research points in this direction to better promote the application of SSMs.
SN - 2097-3330
SN - 2731-9008
ER -