Hybrid transformer and CNN for hyperspectral image classification with spectral band selection
Abstract
Hyperspectral images (HSI) play a vital role in remote sensing applications such as surveillance and precision agriculture, yet they face challenges including limited labeled samples and class imbalance. This paper presents a new method for HSI classification by integrating spectral band selection with advanced deep learning techniques. First, we propose an optimized band selection method alongside data augmentation using crop-and-resize transformations to enhance feature diversity. Second, we introduce a hybrid convolution-transformer framework that combines Vision Transformer (ViT) and 2D-CNN architectures, supplemented with a sequence aggregation layer to mitigate overfitting on small datasets. This hybrid model effectively captures both spatial and spectral features while preserving critical spectral details. Finally, we employ a support vector machine for robust classification. Extensive experiments on three widely used HSI datasets demonstrate that our proposed ViT-CNN-BS framework achieves superior classification accuracy compared to state-of-the-art few-shot learning methods and traditional approaches. Beyond achieving higher accuracy, our approach also reduces computational complexity, making it suitable for resource-constrained applications. The findings highlight the potential of hybrid deep learning frameworks for advancing remote sensing analytics, with possible extensions to environmental monitoring, disaster management, and precision agriculture.
Related
- Near-Infrared Hyperspectral Classification of Waste Textiles Using Complementary Spectral Representations and a Convolution-Enhanced Patch Transformer
- Fine-grained hierarchical crop type classification from integrated hyperspectral EnMAP data and multispectral sentinel-2 time series: A large-scale dataset and dual-stream transformer method
- An explainable hybrid CNN-transformer framework for multi-representation leukaemia classification
- An attention-based CNN–Transformer hybrid framework for osteoporosis and osteopenia classification in knee X-ray images
- Attention-integrated hybrid CNN–vision transformer framework for robust multi-class cervical cytology classification using liquid-based cytology images
BibTeX
@article{abdelhafez2026hybrid,
title = {{Hybrid transformer and CNN for hyperspectral image classification with spectral band selection}},
author = {Eman N. Abdelhafez and Ahmed Hagag and Tamer A. Abassy},
journal = {Scientific Reports},
year = {2026},
month = oct,
doi = {10.1038/s41598-026-72102-z},
url = {https://www.nature.com/articles/s41598-026-72102-z},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Hybrid transformer and CNN for hyperspectral image classification with spectral band selection
AU - Eman N. Abdelhafez
AU - Ahmed Hagag
AU - Tamer A. Abassy
PY - 2026
DA - 2026/10/12
JO - Scientific Reports
DO - 10.1038/s41598-026-72102-z
UR - https://www.nature.com/articles/s41598-026-72102-z
AB - Hyperspectral images (HSI) play a vital role in remote sensing applications such as surveillance and precision agriculture, yet they face challenges including limited labeled samples and class imbalance. This paper presents a new method for HSI classification by integrating spectral band selection with advanced deep learning techniques. First, we propose an optimized band selection method alongside data augmentation using crop-and-resize transformations to enhance feature diversity. Second, we introduce a hybrid convolution-transformer framework that combines Vision Transformer (ViT) and 2D-CNN architectures, supplemented with a sequence aggregation layer to mitigate overfitting on small datasets. This hybrid model effectively captures both spatial and spectral features while preserving critical spectral details. Finally, we employ a support vector machine for robust classification. Extensive experiments on three widely used HSI datasets demonstrate that our proposed ViT-CNN-BS framework achieves superior classification accuracy compared to state-of-the-art few-shot learning methods and traditional approaches. Beyond achieving higher accuracy, our approach also reduces computational complexity, making it suitable for resource-constrained applications. The findings highlight the potential of hybrid deep learning frameworks for advancing remote sensing analytics, with possible extensions to environmental monitoring, disaster management, and precision agriculture.
SN - 2045-2322
ER -