Hierarchical cross-modal spatiotemporal attention network for crop yield prediction in complex Himalayan agroecosystems
4 affiliations
- 1Agricultural Research Information System (ARIS), SKUAST of Kashmir
- 2University of Agricultural Sciences
- 3Faculty of Veterinary Sciences and Animal Husbandry, SKUAST of Kashmir
- 4Centre for Artificial Intelligence and Machine Learning, SKUAST of Kashmir
Abstract
Introduction: Accurate crop yield prediction is essential for precision agriculture, climate-resilient farm management, and timely decision-making. However, yield prediction remains challenging because of complex, nonlinear, and multi-scale interactions among crop physiology, soil conditions, weather variability, and management practices. Conventional statistical, machine learning, and deep learning approaches, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and long short-term memory (LSTM) networks, often have limitations in capturing long-range temporal dependencies, hierarchical spatial relationships, heterogeneous data modalities, and predictive uncertainty.
Methods: This study proposes a Hierarchical Cross-Modal Spatio-Temporal Attention Network (HCM-STAN) for crop yield prediction using a unified attention-based framework for integrating heterogeneous agro-environmental data. The architecture incorporates four principal components: (i) hierarchical spatio-temporal attention operating across tree, block, orchard, and regional scales; (ii) a cross-modal attention module for adaptive fusion of IoT-based soil sensor observations, weather data, plant physiological measurements, and drone-based multispectral imagery; (iii) a soil-aware attention head that explicitly models soil memory, hysteresis effects, and root-zone dynamics; and (iv) an uncertainty-aware prediction module combining Monte Carlo dropout and quantile regression to generate calibrated prediction intervals. The framework was evaluated using a multi-source dataset collected from Himalayan apple orchards during 2023–2026, comprising 1,247 monitored trees across three commercial orchards.
Results: The proposed HCM-STAN achieved an R ² of 0.89 for four-week-ahead crop yield prediction, representing a 15.6% improvement over the CNN-LSTM baseline. Furthermore, zero-shot transfer to mango orchards achieved an R ² of 0.67, corresponding to approximately 75% of the performance obtained under full training.
Discussion: The results demonstrate that integrating hierarchical spatio-temporal representation learning, cross-modal attention, soil-aware modelling, and uncertainty estimation can effectively address the heterogeneous and dynamic nature of crop yield prediction in complex mountain agroecosystems. The observed zero-shot transfer performance further indicates the potential of HCM-STAN to generalize across crop systems without crop-specific retraining. Overall, the proposed framework provides a promising foundation for reliable, uncertainty-aware yield forecasting and can support precision agriculture, climate-smart farm management, and data-driven decision-making in heterogeneous agroecosystems.
Related
- A non-local multi-head spatiotemporal attention LSTM for vehicle trajectory prediction
- HMVA-DDI: A Hierarchical Multi-View Attention Network for Drug-Drug Interaction and Severity Prediction
- PeTMiner: Mining plant endogenous target mimics via ternary interaction modeling with hierarchical attention and adaptive weighted loss
- Cross-attention encoding models reveal dynamic spatiotemporal routing across human higher visual cortex
- CHACAM: a cell-cell interaction-guided hierarchical attention model for high-precision cell identity annotation of scRNA-seq data in early C. elegans embryogenesis
BibTeX
@article{mir2026hierarchical,
title = {{Hierarchical cross-modal spatiotemporal attention network for crop yield prediction in complex Himalayan agroecosystems}},
author = {Shabir Ahmad Mir and Umarfarooque Momin and Sudeep Kumar E. and Itfaq Ahmad Mir and Ab Rouf Khan and G. M. Mir},
journal = {Frontiers in Plant Science},
year = {2026},
month = oct,
doi = {10.3389/fpls.2026.1933033},
url = {https://www.frontiersin.org/articles/10.3389/fpls.2026.1933033/full},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Hierarchical cross-modal spatiotemporal attention network for crop yield prediction in complex Himalayan agroecosystems
AU - Shabir Ahmad Mir
AU - Umarfarooque Momin
AU - Sudeep Kumar E.
AU - Itfaq Ahmad Mir
AU - Ab Rouf Khan
AU - G. M. Mir
PY - 2026
DA - 2026/10/14
JO - Frontiers in Plant Science
DO - 10.3389/fpls.2026.1933033
UR - https://www.frontiersin.org/articles/10.3389/fpls.2026.1933033/full
AB - Introduction: Accurate crop yield prediction is essential for precision agriculture, climate-resilient farm management, and timely decision-making. However, yield prediction remains challenging because of complex, nonlinear, and multi-scale interactions among crop physiology, soil conditions, weather variability, and management practices. Conventional statistical, machine learning, and deep learning approaches, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and long short-term memory (LSTM) networks, often have limitations in capturing long-range temporal dependencies, hierarchical spatial relationships, heterogeneous data modalities, and predictive uncertainty. Methods: This study proposes a Hierarchical Cross-Modal Spatio-Temporal Attention Network (HCM-STAN) for crop yield prediction using a unified attention-based framework for integrating heterogeneous agro-environmental data. The architecture incorporates four principal components: (i) hierarchical spatio-temporal attention operating across tree, block, orchard, and regional scales; (ii) a cross-modal attention module for adaptive fusion of IoT-based soil sensor observations, weather data, plant physiological measurements, and drone-based multispectral imagery; (iii) a soil-aware attention head that explicitly models soil memory, hysteresis effects, and root-zone dynamics; and (iv) an uncertainty-aware prediction module combining Monte Carlo dropout and quantile regression to generate calibrated prediction intervals. The framework was evaluated using a multi-source dataset collected from Himalayan apple orchards during 2023–2026, comprising 1,247 monitored trees across three commercial orchards. Results: The proposed HCM-STAN achieved an R ² of 0.89 for four-week-ahead crop yield prediction, representing a 15.6% improvement over the CNN-LSTM baseline. Furthermore, zero-shot transfer to mango orchards achieved an R ² of 0.67, corresponding to approximately 75% of the performance obtained under full training. Discussion: The results demonstrate that integrating hierarchical spatio-temporal representation learning, cross-modal attention, soil-aware modelling, and uncertainty estimation can effectively address the heterogeneous and dynamic nature of crop yield prediction in complex mountain agroecosystems. The observed zero-shot transfer performance further indicates the potential of HCM-STAN to generalize across crop systems without crop-specific retraining. Overall, the proposed framework provides a promising foundation for reliable, uncertainty-aware yield forecasting and can support precision agriculture, climate-smart farm management, and data-driven decision-making in heterogeneous agroecosystems.
SN - 1664-462X
ER -