Skip to content
DraconiticResearch
Search Sources API
197,787 studies31 sourcesUpdated 17 min ago
Peer-reviewedCC BY 4.0PLOS One

Research on multimodal dense small object detection algorithms guided by cross-modal information

  • Jiacheng Hu1
  • Yumeng Ma2
  • Yue Xing3
  • Yun Zi4
  • Yingnan Deng5
  • Ming Wang6
  • Heyao Liu7
  • Junliang Du
7 affiliations
  1. 1A. B. Freeman School of Business, Tulane University, New Orleans, United States of America
  2. 2Ira A. Fulton School of Engineering, Arizona State University, Tempe, United States of America
  3. 3School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, United States of America
  4. 4School of Computer Science, Georgia Institute of Technology, Atlanta, United States of America
  5. 5College of Graduate and Professional Studies, Trine University, Phoenix, United States of America
  6. 6College of Engineering, Northeastern University, Boston, United States of America
  7. 7MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, China
Open at sourcePDFDOICite

Abstract

To address the challenges of modality heterogeneity, scale inconsistency, and background interference in dense small object detection under multimodal conditions, this paper proposes a novel detection framework based on cross-modality guidance and hierarchical scale refinement. Built upon the RT-DETR backbone, the framework integrates a Cross-Modality Guided Dynamic Fusion (CMG-DF) module, which performs semantic-level recalibration between infrared and visible features via a learnable modality attention mechanism, and a Hierarchical Scale Refinement Network (HSRN), which enhances semantic consistency and boundary continuity across scales through bidirectional residual flow and graph-based relational modeling. To validate the effectiveness of the proposed method, extensive comparison and ablation studies are conducted on two public multimodal benchmarks, SMOD and LLVIP. Experimental results show that the proposed method achieves 92.7% mAP@50 and 69.5% mAP@50:95 on SMOD, as well as 77.3% mAP@50 and 43.1% mAP@50:95 on LLVIP, consistently outperforming existing state-of-the-art multimodal detection algorithms. Qualitative visualizations further confirm the robustness and enhancement capability of the method for small objects under low illumination, occlusion, and complex background conditions, highlighting its strong structural generalization and practical deployment potential.

Related

  1. Decoding object categories from electroencephalography during free viewing reveals early information evolution compared to passive viewing.
    Carmel R Auerbach-Asch, Gal Vishne, Oded Wertheimer +1
    Journal of visionPeer-reviewedCC BY-NC-NDFull textNeuroscienceEurope PMC
  2. Learn the interactions: Weakly supervised video anomaly detection with human-object interactions
    Maowen Zhou, Erma Rahayu Mohd Faizal Abdullah, Aznul Qalid Md Sabri +1
    PLOS OnePeer-reviewedCC BY 4.0Full textNeurosciencePLOS
  3. IV-PAFNet: Infrared-visible cross-modal alignment and fusion network for nighttime UAV object detection
    Cheng Yuan, Shichang Shang, Jinghao Zhang +2
    PLOS OnePeer-reviewedCC BYFull textBioinformatics & computational biologyNeurosciencePLOSEurope PMCPMC
  4. GLACIER: Rethinking Mass Spectrum Prediction as an Object Detection Problem
    Rui-Xi Wang, Runzhong Wang, Connor W. Coley
    arXivPreprintCC BY-NC-ND 4.0Full text1 citedBioinformatics & computational biologyarXiv
  5. Evaluation of object detection neural networks to find topologically associated domains in bulk Hi-C data
    N. Philipp, C. L. Gruennagel, S. Janssen +1
    bioRxivPreprintCC BY-NCFull textBioinformatics & computational biologybioRxiv

Cite

BibTeXRIS
BibTeX
@article{hu2026research,
  title = {{Research on multimodal dense small object detection algorithms guided by cross-modal information}},
  author = {Jiacheng Hu and Yumeng Ma and Yue Xing and Yun Zi and Yingnan Deng and Ming Wang and Heyao Liu and Junliang Du},
  journal = {PLOS One},
  year = {2026},
  month = sep,
  doi = {10.1371/journal.pone.0348533},
  url = {https://doi.org/10.1371/journal.pone.0348533},
  note = {CC BY 4.0},
}
RIS
TY  - JOUR
TI  - Research on multimodal dense small object detection algorithms guided by cross-modal information
AU  - Jiacheng Hu
AU  - Yumeng Ma
AU  - Yue Xing
AU  - Yun Zi
AU  - Yingnan Deng
AU  - Ming Wang
AU  - Heyao Liu
AU  - Junliang Du
PY  - 2026
DA  - 2026/09/08
JO  - PLOS One
DO  - 10.1371/journal.pone.0348533
UR  - https://doi.org/10.1371/journal.pone.0348533
AB  - To address the challenges of modality heterogeneity, scale inconsistency, and background interference in dense small object detection under multimodal conditions, this paper proposes a novel detection framework based on cross-modality guidance and hierarchical scale refinement. Built upon the RT-DETR backbone, the framework integrates a Cross-Modality Guided Dynamic Fusion (CMG-DF) module, which performs semantic-level recalibration between infrared and visible features via a learnable modality attention mechanism, and a Hierarchical Scale Refinement Network (HSRN), which enhances semantic consistency and boundary continuity across scales through bidirectional residual flow and graph-based relational modeling. To validate the effectiveness of the proposed method, extensive comparison and ablation studies are conducted on two public multimodal benchmarks, SMOD and LLVIP. Experimental results show that the proposed method achieves 92.7% mAP@50 and 69.5% mAP@50:95 on SMOD, as well as 77.3% mAP@50 and 43.1% mAP@50:95 on LLVIP, consistently outperforming existing state-of-the-art multimodal detection algorithms. Qualitative visualizations further confirm the robustness and enhancement capability of the method for small objects under low illumination, occlusion, and complex background conditions, highlighting its strong structural generalization and practical deployment potential.
SN  - 1932-6203
ER  -