Hybrid hypergraph networks for multimodal sequence data classification
Abstract
Modeling temporal multimodal data requires both long-range temporal dependencies and cross-modal interactions to be represented. Existing approaches commonly model each modality separately and then apply shallow fusion, which can limit the representation of higher-order temporal structure. We propose a hybrid hypergraph network (HHN) that combines an established order-preserving alignment for candidate cross-modal links with an entropy-guided mechanism for constructing intra-modal hyperedges. Timestamped segments are represented as nodes; within each modality, an adaptive temporal window and a maximum-entropy-difference rule form higher-order hyperedges that are aggregated by hypergraph convolution. Across modalities, temporally weighted pairwise links and graph attention provide cross-modal fusion. Across four processed multimodal datasets, the reported mean average precision and area under the receiver operating characteristic curve were higher for HHN than for the evaluated baselines. Ablation results further indicate that both the hybrid graph modules and multimodal inputs contribute to classification performance.
Related
- Relation-Aware Graph-Hypergraph Networks for Antibody Sequence-Structure Co-Design
- SynerHyper: Predicting Anticancer Drug Synergy for Lung Cancer Using Uni-Mol Enhanced Hypergraph Neural Networks and Multimodal Fusion.
- Hybrid Multimodal Fusion of Raw ECG Signals and Derived Measurements for Stroke Classification
- Hybrid deep learning and meta learning for Alzheimer disease classification via multimodal radiomic and visual features
- Quantum Entangled Multimodal Fusion Networks (QEMFN): Resource-Aware Hybrid Vision-Language Fusion via Trainable Entanglement
BibTeX
@article{xu2026hybrid,
title = {{Hybrid hypergraph networks for multimodal sequence data classification}},
author = {Feng Xu and Zhifeng Zheng and Hui Wang and Yuting Huang and Danwei Zhang and Zizhu Fan},
journal = {Scientific Reports},
year = {2026},
month = oct,
doi = {10.1038/s41598-026-73391-0},
url = {https://www.nature.com/articles/s41598-026-73391-0},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Hybrid hypergraph networks for multimodal sequence data classification
AU - Feng Xu
AU - Zhifeng Zheng
AU - Hui Wang
AU - Yuting Huang
AU - Danwei Zhang
AU - Zizhu Fan
PY - 2026
DA - 2026/10/09
JO - Scientific Reports
DO - 10.1038/s41598-026-73391-0
UR - https://www.nature.com/articles/s41598-026-73391-0
AB - Modeling temporal multimodal data requires both long-range temporal dependencies and cross-modal interactions to be represented. Existing approaches commonly model each modality separately and then apply shallow fusion, which can limit the representation of higher-order temporal structure. We propose a hybrid hypergraph network (HHN) that combines an established order-preserving alignment for candidate cross-modal links with an entropy-guided mechanism for constructing intra-modal hyperedges. Timestamped segments are represented as nodes; within each modality, an adaptive temporal window and a maximum-entropy-difference rule form higher-order hyperedges that are aggregated by hypergraph convolution. Across modalities, temporally weighted pairwise links and graph attention provide cross-modal fusion. Across four processed multimodal datasets, the reported mean average precision and area under the receiver operating characteristic curve were higher for HHN than for the evaluated baselines. Ablation results further indicate that both the hybrid graph modules and multimodal inputs contribute to classification performance.
SN - 2045-2322
ER -