Hallucination aware hypergraph memory for few shot RGB D anomaly detection

Abstract

Cross-modal inconsistency is a major challenge for reliable RGB-D anomaly detection, especially under few-shot settings where normal multimodal patterns are insufficiently represented. In this work, we study hallucination in a restricted perception-level sense, referring to unreliable anomaly responses induced by inconsistent or insufficiently grounded multimodal evidence. Unlike textual hallucinations in LLMs/VLMs, the phenomenon considered here occurs in multimodal perception, where conflicting RGB appearance and depth geometry may lead to false defect alarms, missed anomalies, or inaccurate localization maps. To address this problem, we introduce Hallucination-Aware Hypergraph Memory (HA-HM), a framework that models high-order cross-modal consistency through a unified, learnable hypergraph. HA-HM integrates four synergistic components: a Learnable Hypergraph Constructor for adaptive multimodal grouping, a Consistency-Guided Memory Prototype Learner that stores high-fidelity normal prototypes, an Iterative Hypergraph Refiner for prototype-guided inconsistency correction, and a Hallucination Localization module for dense anomaly localization. Evaluations on the MVTec 3D-AD and Eyecandies RGB-D benchmarks demonstrate the effectiveness of HA-HM in few-shot multimodal anomaly detection and localization. Under challenging 1-, 2-, and 4-shot conditions, HA-HM achieves consistent improvements over existing training-based and training-free methods in image-level AUROC, pixel-level AUROC, and AUPRO. Ablation studies further validate the contribution of each component. These results suggest that explicitly modeling cross-modal consistency can improve the reliability of RGB-D anomaly detection systems.

Spatially Context-Aware Transformers Facilitate Modeling-Based Anomaly Detection of Subtle Lesions in Brain MRI Images
J. Schwarz, L. Will, J. Wellmer +1
medRxiv14 Sep 2026PreprintCC BYFull textEpidemiology & public healthmedRxiv
HalluciGuard-eye: knowledge graph-enhanced retrieval-augmented generation for hallucination detection and mitigation in ophthalmic multimodal large language models
Ge Wu, Chong Ai, Weiqi Wu
Frontiers in Endocrinology20 Oct 2026Peer-reviewedCC BY 4.0Full textEndocrinology & metabolismCrossrefEurope PMCPMC
Hybrid quantum-classical machine learning for industrial multi-anomaly detection via single acoustic sensor
Takao Tomono, Kazuya Tsujimura
Scientific Reports1 Jul 2026Peer-reviewedCC BYFull textGeneral medicineDOAJ
Cross-lineage anomaly detection of microRNA expression: a framework for deriving exploratory candidate biomarkers in sarcoma.
Mritunjoy Dey, Piotr Remiszewski, Ewelina Perdas +3
Frontiers in genetics12 Oct 2026Peer-reviewedCC BYFull textOncologyEurope PMC
Dual-domain self-supervised feature alignment via spectral–spatial representation learning for deepfake anomaly detection
Yi He
PLOS One11 Sep 2026Peer-reviewedCC BY 4.0Full textBioinformatics & computational biology ProbabilityPLOS

Cite

BibTeX RIS

BibTeX

@article{liu2026hallucination,
  title = {{Hallucination aware hypergraph memory for few shot RGB D anomaly detection}},
  author = {Jiayi Liu and Xiyue Fan and Xin Peng and Yixin Xu and Yi Wang},
  journal = {Scientific Reports},
  year = {2026},
  month = sep,
  doi = {10.1038/s41598-026-66078-z},
  url = {https://doi.org/10.1038/s41598-026-66078-z},
  note = {CC BY},
}

RIS

TY  - JOUR
TI  - Hallucination aware hypergraph memory for few shot RGB D anomaly detection
AU  - Jiayi Liu
AU  - Xiyue Fan
AU  - Xin Peng
AU  - Yixin Xu
AU  - Yi Wang
PY  - 2026
DA  - 2026/09/01
JO  - Scientific Reports
DO  - 10.1038/s41598-026-66078-z
UR  - https://doi.org/10.1038/s41598-026-66078-z
AB  - Cross-modal inconsistency is a major challenge for reliable RGB-D anomaly detection, especially under few-shot settings where normal multimodal patterns are insufficiently represented. In this work, we study hallucination in a restricted perception-level sense, referring to unreliable anomaly responses induced by inconsistent or insufficiently grounded multimodal evidence. Unlike textual hallucinations in LLMs/VLMs, the phenomenon considered here occurs in multimodal perception, where conflicting RGB appearance and depth geometry may lead to false defect alarms, missed anomalies, or inaccurate localization maps. To address this problem, we introduce Hallucination-Aware Hypergraph Memory (HA-HM), a framework that models high-order cross-modal consistency through a unified, learnable hypergraph. HA-HM integrates four synergistic components: a Learnable Hypergraph Constructor for adaptive multimodal grouping, a Consistency-Guided Memory Prototype Learner that stores high-fidelity normal prototypes, an Iterative Hypergraph Refiner for prototype-guided inconsistency correction, and a Hallucination Localization module for dense anomaly localization. Evaluations on the MVTec 3D-AD and Eyecandies RGB-D benchmarks demonstrate the effectiveness of HA-HM in few-shot multimodal anomaly detection and localization. Under challenging 1-, 2-, and 4-shot conditions, HA-HM achieves consistent improvements over existing training-based and training-free methods in image-level AUROC, pixel-level AUROC, and AUPRO. Ablation studies further validate the contribution of each component. These results suggest that explicitly modeling cross-modal consistency can improve the reliability of RGB-D anomaly detection systems.
SN  - 2045-2322
ER  -

Abstract

Related

Cite