Evaluating Transferability and Validation Protocols of Convolutional Neural Networks for Acoustic Water Leak Detection Using SebaKMT Logger Recordings
- 1Rivixi LLC
Abstract
<div class="leading-relaxed select-text text-sm flex flex-col [&_strong]:font-semibold [&_b]:font-semibold [&_code:not(pre_code)]:bg-muted [&_code:not(pre_code)]:text-code [&_code:not(pre_code)]:rounded [&_code:not(pre_code)]:px-1 [&_code:not(pre_code)]:py-0.5 [&_code:not(pre_code)]:text-[0.9em] [&_summary]:cursor-pointer [&_h1]:text-xl [&_h2]:text-lg [&_h3]:text-base [&_h4]:text-sm [&_h1]:font-semibold [&_h2]:font-semibold [&_h3]:font-medium [&_h4]:font-medium [&_h1]:!mb-2 [&_h1]:!mt-6 [&_h2]:!mb-2 [&_h2]:!mt-6 [&_h3]:!mb-1 [&_h3]:!mt-3 [&_h4]:!mb-1 [&_h4]:!mt-3 [&_li.task-list-item]:list-none [&_ul:has(li.task-list-item)]:list-none [&_ul]:my-2 [&_ol]:my-2 [&>ul.contains-task-list]:pl-0 [&>ol.contains-task-list]:pl-0 [&_ul]:pl-10 [&_ol]:pl-10 [&>ul:not(.contains-task-list)]:!pl-4 [&>ol:not(.contains-task-list)]:!pl-4 [&_ul]:list-disc [&_ol]:list-decimal [&_li.task-checked>.task-checked-content]:opacity-50 md-divider-spacing break-words gap-2"> This preprint evaluates direct transfer, supervised target adaptation, and validation protocol design of convolutional neural networks for acoustic water leak detection using field recordings from SebaKMT correlation loggers. A single-scale CNN (25,633 parameters) is compared with a three-branch multiscale time–frequency network (TFCNN, 76,769 parameters) and classical machine learning baselines across multiple experimental protocols. In direct transfer from an unstructured multi-instrument collection, multiscale TFCNN increased sensitivity from 78.20% to 86.40% compared to the single-scale network, but exhibited low specificity (16.50%). While supervised adaptation of the output classifier improved test score ranking (ROC-AUC 94.16%), threshold selection subject to high validation recall retained 20 alarms among 40 negative field recordings. When trained from scratch on 180 SebaKMT recordings, random file partitioning yielded 100% accuracy in all ten runs, which is shown to be inflated by shared recording sessions between training and test sets. Under site-disjoint grouped cross-validation across six registered site groups, TFCNN out-of-fold accuracy was 87.83 ± 3.22% (mean within-fold ROC-AUC 94.28 ± 3.87%). A classical logistic regression baseline using nine handcrafted signal descriptors without digital amplitude achieved an accuracy similar to TFCNN (87.78% vs. 87.83%), with higher sensitivity (98.33% vs. 92.67%) and lower specificity (82.50% vs. 85.42%), demonstrating no detectable accuracy advantage for the deep architecture on this corpus. Furthermore, logistic regression on digital RMS alone reached 94.44% accuracy at threshold 0.50 (within-fold ROC-AUC 99.67 ± 0.58%), although its sensitivity was lower at 83.33%. This strong predictive association motivates investigation of acquisition-related shortcuts rather than confirming a physical acoustic mechanism. The study concludes that acoustic diagnostics on small field datasets require strict site-disjoint validation protocols and transparent classical feature baselines alongside deep neural networks to prevent session leakage and shortcut learning.
Related
- Evaluating Pedestrian Crossing Safety: Implementing and Evaluating a Convolutional Neural Network Model Trained on Paired Aerial and Subjective Perspective Images (Vol 11, E42428, 2025)
- Transferability of Transcriptome-Based Gene-Dependency Models from Two-Dimensional Cancer Models to Patient-Derived Organoids: Blinded External Validation
- The ClimaSentAR dataset for evaluating cross-domain transferability of large language models in climate sentiment analysis
- Validation-Aware VNIR Spectroscopy Reveals Target-Specific Transferability of SOC, SIC, and Gypsum Predictions in Dryland Agricultural Soils
- Evaluating the Transferability of Pathology Foundation Models Across Cancer-related H&E Neurodegeneration-related Immunohistochemical Classification Tasks
BibTeX
@misc{ivanaiskii2026evaluating,
title = {{Evaluating Transferability and Validation Protocols of Convolutional Neural Networks for Acoustic Water Leak Detection Using SebaKMT Logger Recordings}},
author = {Aleksandr Ivanaiskii and Evgeny Ivanaiskii and Sergei Shipilov},
howpublished = {Zenodo},
year = {2026},
month = oct,
doi = {10.5281/zenodo.23446900},
url = {https://zenodo.org/records/23446900},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Evaluating Transferability and Validation Protocols of Convolutional Neural Networks for Acoustic Water Leak Detection Using SebaKMT Logger Recordings
AU - Aleksandr Ivanaiskii
AU - Evgeny Ivanaiskii
AU - Sergei Shipilov
PY - 2026
DA - 2026/10/18
PB - Zenodo
DO - 10.5281/zenodo.23446900
UR - https://zenodo.org/records/23446900
AB - <div class="leading-relaxed select-text text-sm flex flex-col [&_strong]:font-semibold [&_b]:font-semibold [&_code:not(pre_code)]:bg-muted [&_code:not(pre_code)]:text-code [&_code:not(pre_code)]:rounded [&_code:not(pre_code)]:px-1 [&_code:not(pre_code)]:py-0.5 [&_code:not(pre_code)]:text-[0.9em] [&_summary]:cursor-pointer [&_h1]:text-xl [&_h2]:text-lg [&_h3]:text-base [&_h4]:text-sm [&_h1]:font-semibold [&_h2]:font-semibold [&_h3]:font-medium [&_h4]:font-medium [&_h1]:!mb-2 [&_h1]:!mt-6 [&_h2]:!mb-2 [&_h2]:!mt-6 [&_h3]:!mb-1 [&_h3]:!mt-3 [&_h4]:!mb-1 [&_h4]:!mt-3 [&_li.task-list-item]:list-none [&_ul:has(li.task-list-item)]:list-none [&_ul]:my-2 [&_ol]:my-2 [&>ul.contains-task-list]:pl-0 [&>ol.contains-task-list]:pl-0 [&_ul]:pl-10 [&_ol]:pl-10 [&>ul:not(.contains-task-list)]:!pl-4 [&>ol:not(.contains-task-list)]:!pl-4 [&_ul]:list-disc [&_ol]:list-decimal [&_li.task-checked>.task-checked-content]:opacity-50 md-divider-spacing break-words gap-2"> This preprint evaluates direct transfer, supervised target adaptation, and validation protocol design of convolutional neural networks for acoustic water leak detection using field recordings from SebaKMT correlation loggers. A single-scale CNN (25,633 parameters) is compared with a three-branch multiscale time–frequency network (TFCNN, 76,769 parameters) and classical machine learning baselines across multiple experimental protocols. In direct transfer from an unstructured multi-instrument collection, multiscale TFCNN increased sensitivity from 78.20% to 86.40% compared to the single-scale network, but exhibited low specificity (16.50%). While supervised adaptation of the output classifier improved test score ranking (ROC-AUC 94.16%), threshold selection subject to high validation recall retained 20 alarms among 40 negative field recordings. When trained from scratch on 180 SebaKMT recordings, random file partitioning yielded 100% accuracy in all ten runs, which is shown to be inflated by shared recording sessions between training and test sets. Under site-disjoint grouped cross-validation across six registered site groups, TFCNN out-of-fold accuracy was 87.83 ± 3.22% (mean within-fold ROC-AUC 94.28 ± 3.87%). A classical logistic regression baseline using nine handcrafted signal descriptors without digital amplitude achieved an accuracy similar to TFCNN (87.78% vs. 87.83%), with higher sensitivity (98.33% vs. 92.67%) and lower specificity (82.50% vs. 85.42%), demonstrating no detectable accuracy advantage for the deep architecture on this corpus. Furthermore, logistic regression on digital RMS alone reached 94.44% accuracy at threshold 0.50 (within-fold ROC-AUC 99.67 ± 0.58%), although its sensitivity was lower at 83.33%. This strong predictive association motivates investigation of acquisition-related shortcuts rather than confirming a physical acoustic mechanism. The study concludes that acoustic diagnostics on small field datasets require strict site-disjoint validation protocols and transparent classical feature baselines alongside deep neural networks to prevent session leakage and shortcut learning.
ER -