Lightweight multimodal fusion for UAV classification with explicit missing-modality modeling
Abstract
Fine-grained unmanned aerial vehicle (UAV) type classification benefits from complementary sensors, yet real deployments face incomplete observations and heterogeneous stream reliability. We present a missing-aware fusion framework for RGB detections (YOLO11-S), Livox Avia LiDAR, 360° LiDAR, and radar-enhanced point clouds on MMAUD. Confidence–density scores are masked and renormalized so fusion weight is assigned only to observed streams (Eq. (2)); each absent stream still occupies a classifier slot via a learnable missing embedding that is not multiplied by the zero weight (Eq. (1)), keeping absence as a trainable signal rather than a numerical zero. On a time-stratified split, Tiny MLP (≈ 0.1 M fusion parameters) reaches 96.12% test accuracy in a representative seed-42 run. Over seeds 42–46, final test accuracy is 95.29 ± 0.60% without training-time modality dropout and 95.77 ± 0.90% with 30% training ModDrop (mean ± sample s.d.). Under matched 30% test missingness, accuracy remains 83.68% (seed 42). Protocol-aligned baselines and orthogonal ablations support the slot rule. Fusion-stage inference is about 2.1 ms per sample with pre-extracted features on an RTX 4090. Under our fixed split and frozen features—not as a challenge-leaderboard claim—compact missing-aware fusion is practical for heterogeneous UAV sensing under partial sensor availability.
Related
- Multimodal classification of skin lesions based on lightweight gated low-rank hyper-fusion network
- A single-cell multimodal foundation model enables cross modality analysis between transcriptome and chromatin
- Artificial Intelligence Across the Cardiac Amyloidosis Diagnostic and Management Pathway: From Single-Modality Detection to Multimodal Clinical Integration
- Interpretable and lightweight multimodal Transformer assessment of English education ecosystem health
- Deployment of edge-lightweight AI vertical models in multimodal anomaly detection and energy optimization of tech-long production line
BibTeX
@article{feng2026lightweight,
title = {{Lightweight multimodal fusion for UAV classification with explicit missing-modality modeling}},
author = {Yibo Feng and Ning Li and Di Wu and Ximing Ma and Xiangyue Yu and Haiyang Bao},
journal = {Scientific Reports},
year = {2026},
month = oct,
doi = {10.1038/s41598-026-76332-z},
url = {https://www.nature.com/articles/s41598-026-76332-z},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Lightweight multimodal fusion for UAV classification with explicit missing-modality modeling
AU - Yibo Feng
AU - Ning Li
AU - Di Wu
AU - Ximing Ma
AU - Xiangyue Yu
AU - Haiyang Bao
PY - 2026
DA - 2026/10/14
JO - Scientific Reports
DO - 10.1038/s41598-026-76332-z
UR - https://www.nature.com/articles/s41598-026-76332-z
AB - Fine-grained unmanned aerial vehicle (UAV) type classification benefits from complementary sensors, yet real deployments face incomplete observations and heterogeneous stream reliability. We present a missing-aware fusion framework for RGB detections (YOLO11-S), Livox Avia LiDAR, 360° LiDAR, and radar-enhanced point clouds on MMAUD. Confidence–density scores are masked and renormalized so fusion weight is assigned only to observed streams (Eq. (2)); each absent stream still occupies a classifier slot via a learnable missing embedding that is not multiplied by the zero weight (Eq. (1)), keeping absence as a trainable signal rather than a numerical zero. On a time-stratified split, Tiny MLP (≈ 0.1 M fusion parameters) reaches 96.12% test accuracy in a representative seed-42 run. Over seeds 42–46, final test accuracy is 95.29 ± 0.60% without training-time modality dropout and 95.77 ± 0.90% with 30% training ModDrop (mean ± sample s.d.). Under matched 30% test missingness, accuracy remains 83.68% (seed 42). Protocol-aligned baselines and orthogonal ablations support the slot rule. Fusion-stage inference is about 2.1 ms per sample with pre-extracted features on an RTX 4090. Under our fixed split and frozen features—not as a challenge-leaderboard claim—compact missing-aware fusion is practical for heterogeneous UAV sensing under partial sensor availability.
SN - 2045-2322
ER -