Skip to content
DraconiticResearch
Search Sources API
203,234 studies31 sourcesUpdated 51 min ago
Peer-reviewedCC BY 4.0PLOS One

State-aware adaptive multimodal fusion for ultra-short-term solar irradiance forecasting using sky cloud images

  • Houhang Zhao1
  • Aiyi Song2
  • Yu Jin1
  • Haoxiang Yu3
  1. 1School of Statistics, Dongbei University of Finance and Economics, Dalian, China
  2. 2School of Health and Medical Technology, Dalian Neusoft University of Information, Dalian, China
  3. 3Academy for Educational Development and Innovation, The Education University of Hong Kong, Hong Kong, China
Read full textDOICite

Abstract

As renewable energy penetration continues to increase, accurate ultra-short-term solar irradiance forecasting is essential for improving energy utilization and supporting power system operation. However, irradiance variations are jointly influenced by historical trends and rapid cloud changes, making traditional single-modal methods insufficient for capturing complex dynamics under varying weather conditions. This paper proposes a multimodal solar irradiance forecasting framework based on the collaborative representation of sky cloud images and numerical sequences. The framework introduces a heterogeneous modal state encoding mechanism and a state-aware adaptive selective attention fusion mechanism to enhance interactions between visual weather information and historical physical patterns. Specifically, the numerical modality captures irradiance trends and temporal correlations through dependency modeling, while the image modality extracts cloud structures and evolution features through spatiotemporal representation. The proposed fusion mechanism dynamically adjusts modality contributions according to forecasting conditions, enabling effective integration of complementary information. Experimental results demonstrate that the proposed framework outperforms benchmark models in forecasting accuracy and robustness, validating the effectiveness of combining visual cloud information with numerical evolution patterns for ultra-short-term solar irradiance forecasting.

Related

  1. PROSWIN: Probabilistic Solar Wind Speed Forecasting Using Deep Distributional Regression From Solar Images
    Daniel Collin, Yuri Shprits, Luca Chiarabini +3
    AGU AdvancesPeer-reviewedCC BY 4.0Full textGeneral earth scienceCrossref
  2. An interpretable model for efficient and robust runoff forecasting via causal-linear attention and adaptive KAN decoding
    Wei-can Tian, Lin Cheng, Wen-chuan Wang
    Scientific ReportsPeer-reviewedCC BY-NC-ND 4.0Full textEnvironmental science & pollutionCrossref
  3. An intelligent weighted adaptive regression process for reliable and uncertainty aware drought forecasting
    Heming Jia, Iman Ahmadianfar, Masoud Karbasi +1
    iSciencePeer-reviewedCC BY-NC-ND 4.0Full textEnvironmental science & pollutionCell PressCrossrefEurope PMCPMC
  4. Bi-Mamba-Load: A bidirectional structured state space model with sequence relationship adaptive perception for electricity load forecasting
    Junwei Zhu, Lingzhi Xi, Chongyang Liao +3
    PLOS OnePeer-reviewedCC BYFull textNeuroscienceAtmospheric science & meteorologyPLOSEurope PMCPMC
  5. Multimodal Interaction and Residual Fusion for Cross-Well Acoustic-Impedance Reconstruction
    Xue Rui, Weina Zhang, Jing Ren +2
    Applied SciencesPeer-reviewedCC BYFull textGeophysics & seismologyDOAJ

Cite

BibTeXRIS
BibTeX
@article{zhao2026stateaware,
  title = {{State-aware adaptive multimodal fusion for ultra-short-term solar irradiance forecasting using sky cloud images}},
  author = {Houhang Zhao and Aiyi Song and Yu Jin and Haoxiang Yu},
  journal = {PLOS One},
  year = {2026},
  month = nov,
  doi = {10.1371/journal.pone.0362193},
  url = {https://doi.org/10.1371/journal.pone.0362193},
  note = {CC BY 4.0},
}
RIS
TY  - JOUR
TI  - State-aware adaptive multimodal fusion for ultra-short-term solar irradiance forecasting using sky cloud images
AU  - Houhang Zhao
AU  - Aiyi Song
AU  - Yu Jin
AU  - Haoxiang Yu
PY  - 2026
DA  - 2026/11/03
JO  - PLOS One
DO  - 10.1371/journal.pone.0362193
UR  - https://doi.org/10.1371/journal.pone.0362193
AB  - As renewable energy penetration continues to increase, accurate ultra-short-term solar irradiance forecasting is essential for improving energy utilization and supporting power system operation. However, irradiance variations are jointly influenced by historical trends and rapid cloud changes, making traditional single-modal methods insufficient for capturing complex dynamics under varying weather conditions. This paper proposes a multimodal solar irradiance forecasting framework based on the collaborative representation of sky cloud images and numerical sequences. The framework introduces a heterogeneous modal state encoding mechanism and a state-aware adaptive selective attention fusion mechanism to enhance interactions between visual weather information and historical physical patterns. Specifically, the numerical modality captures irradiance trends and temporal correlations through dependency modeling, while the image modality extracts cloud structures and evolution features through spatiotemporal representation. The proposed fusion mechanism dynamically adjusts modality contributions according to forecasting conditions, enabling effective integration of complementary information. Experimental results demonstrate that the proposed framework outperforms benchmark models in forecasting accuracy and robustness, validating the effectiveness of combining visual cloud information with numerical evolution patterns for ultra-short-term solar irradiance forecasting.
SN  - 1932-6203
ER  -