A RAG-enhanced multi-agent LLM/VLM framework for auditable and region-aware structural health monitoring of a historic masonry structure
Abstract
Historic masonry structures require monitoring workflows that are non-destructive, interpretable, and robust to field variability. This paper presents a proof-of-concept, RAG-enhanced structural health monitoring (SHM) workflow demonstrated on a single historic masonry minaret case study, in which a manager–orchestrator coordinates signal analysis, visual analysis, and cross-modal fusion. Vibration data from four sensors installed on this minaret are processed through a reproducible pipeline that includes data cleaning, PSD-based baseline characterisation, adaptive windowing, robust normalisation, and anomaly screening with a 1D convolutional autoencoder trained on healthy reference data (Measurements 1–12), followed by evaluation on Measurements 13–20. Localisation is strengthened through relation-based autoencoders on sensor pairs and sensor-to-zone damage mapping. In parallel, a VLM analyses sensor-linked inspection images from the same case study, while a curated retrieval-augmented structural memory package for the monitored minaret (geometry, materials, baseline dynamics, FE notes, and inspection playbook) supports context-aware interpretation. A three-agent decision layer (Agent A: LLM+RAG, Agent B: VLM+RAG, Agent C: fusion + RAG) reconciles agreements and conflicts between signal- and image-derived evidence to produce case-specific global and zone-level risk assessments. The study compares with-RAG and without-RAG analysis modes within this single deployment and reports structured, traceable outputs, including JSON artefacts, per-image findings, per-sensor summaries, narrative reports, and final risk maps. Its main contribution is to demonstrate how signal-derived anomalies, image-derived evidence, and retrieval-grounded structural memory can be combined within a traceable multi-agent SHM workflow for one monitored historic masonry minaret.
Related
- An attention-enhanced residual network framework for structural damage detection based on structural health monitoring data
- Sparse defect localization - A Periodic Empirical Operator Framework for Inverse Scattering and Structural Health Monitoring
- Structural drivers of persistent CK-MB testing in the troponin era and a nationwide monitoring framework for cardiac biomarker stewardship
- The Self-and-Mutual Structure Model: A General Computational Framework for Adaptive Structural Organization and Reorganization in Multiple Large Language Models
- A Conditional, Boundary-Aware, and Auditable Deletion-Effect Weighting Framework for MCDM with Positive Decision Matrices and a Fixed Set of Alternatives
BibTeX
@article{hacıefendioglu2026ragenhanced,
title = {{A RAG-enhanced multi-agent LLM/VLM framework for auditable and region-aware structural health monitoring of a historic masonry structure}},
author = {Kemal Hacıefendioğlu and Volkan Kahya and Tunahan Aslan},
journal = {Scientific Reports},
year = {2026},
month = oct,
doi = {10.1038/s41598-026-57424-2},
url = {https://www.nature.com/articles/s41598-026-57424-2},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - A RAG-enhanced multi-agent LLM/VLM framework for auditable and region-aware structural health monitoring of a historic masonry structure
AU - Kemal Hacıefendioğlu
AU - Volkan Kahya
AU - Tunahan Aslan
PY - 2026
DA - 2026/10/15
JO - Scientific Reports
DO - 10.1038/s41598-026-57424-2
UR - https://www.nature.com/articles/s41598-026-57424-2
AB - Historic masonry structures require monitoring workflows that are non-destructive, interpretable, and robust to field variability. This paper presents a proof-of-concept, RAG-enhanced structural health monitoring (SHM) workflow demonstrated on a single historic masonry minaret case study, in which a manager–orchestrator coordinates signal analysis, visual analysis, and cross-modal fusion. Vibration data from four sensors installed on this minaret are processed through a reproducible pipeline that includes data cleaning, PSD-based baseline characterisation, adaptive windowing, robust normalisation, and anomaly screening with a 1D convolutional autoencoder trained on healthy reference data (Measurements 1–12), followed by evaluation on Measurements 13–20. Localisation is strengthened through relation-based autoencoders on sensor pairs and sensor-to-zone damage mapping. In parallel, a VLM analyses sensor-linked inspection images from the same case study, while a curated retrieval-augmented structural memory package for the monitored minaret (geometry, materials, baseline dynamics, FE notes, and inspection playbook) supports context-aware interpretation. A three-agent decision layer (Agent A: LLM+RAG, Agent B: VLM+RAG, Agent C: fusion + RAG) reconciles agreements and conflicts between signal- and image-derived evidence to produce case-specific global and zone-level risk assessments. The study compares with-RAG and without-RAG analysis modes within this single deployment and reports structured, traceable outputs, including JSON artefacts, per-image findings, per-sensor summaries, narrative reports, and final risk maps. Its main contribution is to demonstrate how signal-derived anomalies, image-derived evidence, and retrieval-grounded structural memory can be combined within a traceable multi-agent SHM workflow for one monitored historic masonry minaret.
SN - 2045-2322
ER -