A lightweight knowledge distillation framework for few shot unsupervised industrial surface anomaly detection and localization
Abstract
Industrial surface anomaly detection under data-scarce, unsupervised conditions requires models that are both label-efficient and computationally lightweight. This paper presents LKDF-AD, a knowledge-distillation framework combining a frozen pretrained teacher with a compact student network (385,168 parameters), trained using only k = 1–10 normal reference images per category, without any labeled anomalies. Evaluated across 15 MVTec AD categories and five shot counts, the framework achieves 77.88–86.19% image-level and 90.59–94.87% pixel-level AUROC, and generalizes to the independent BTAD benchmark (73.6–90.99% image-level AUROC), with a larger BTAD pixel-AUROC/PRO gap flagged as an open question. Measured end-to-end (teacher plus student together, as deployed), the framework uses 1.93× fewer parameters (11.56 M vs. 22.35 M) and achieves 1.57× lower inference latency (4.436 ms vs. 6.954 ms) than a full-size STFPM-style baseline trained under an identical protocol. Under category-level paired testing (using 15 independent category means, the statistically appropriate unit given repeated measurements across shot counts), neither the image-level nor the pixel-level advantage over STFPM reaches statistical significance (two-sided p = 0.7197 and p = 0.188, respectively). Consequently, the framework’s established advantage over STFPM is its directly measured computational efficiency (1.93× fewer end-to-end parameters, 1.57× lower end-to-end latency), rather than accuracy. In contrast, against the RegAD-inspired baseline, both accuracy advantages remain highly significant under the same corrected test (two-sided p = 0.00201 for image-level and p = 0.00153 for pixel-level). A random, completely untrained student achieves only near-chance performance (52.7% image-level, 46.6% pixel-level AUROC), confirming that the observed localization signal reflects genuine distillation rather than an architectural artifact. A dedicated investigation of the framework’s weakest category (screw, near-chance performance) tested a rotation-sensitivity hypothesis directly across the full test set; the result was a significant correlation in the direction opposite to that hypothesized, and we report the mechanism as unresolved rather than settled. These results position the framework as a genuinely lightweight, empirically characterized — rather than definitively superior — approach to few-shot unsupervised industrial anomaly detection.
Related
- Lightweight anomaly detection via knowledge distillation from ensemble teachers
- Knowledge distillation-based lightweight generative large language models for standardized quality control of Chinese radiology reports
- A Lightweight Universal Machine-Learning Interatomic Potential via Knowledge Distillation for Scalable Atomistic Simulations
- Standard and decoupled knowledge distillation for lightweight radiographic knee osteoarthritis grading and triage
- A temperature-adaptive logit-standardized knowledge distillation for lightweight network intrusion detection
BibTeX
@article{gao2026lightweight,
title = {{A lightweight knowledge distillation framework for few shot unsupervised industrial surface anomaly detection and localization}},
author = {Xian Gao},
journal = {Scientific Reports},
year = {2026},
month = oct,
doi = {10.1038/s41598-026-74501-8},
url = {https://www.nature.com/articles/s41598-026-74501-8},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - A lightweight knowledge distillation framework for few shot unsupervised industrial surface anomaly detection and localization
AU - Xian Gao
PY - 2026
DA - 2026/10/15
JO - Scientific Reports
DO - 10.1038/s41598-026-74501-8
UR - https://www.nature.com/articles/s41598-026-74501-8
AB - Industrial surface anomaly detection under data-scarce, unsupervised conditions requires models that are both label-efficient and computationally lightweight. This paper presents LKDF-AD, a knowledge-distillation framework combining a frozen pretrained teacher with a compact student network (385,168 parameters), trained using only k = 1–10 normal reference images per category, without any labeled anomalies. Evaluated across 15 MVTec AD categories and five shot counts, the framework achieves 77.88–86.19% image-level and 90.59–94.87% pixel-level AUROC, and generalizes to the independent BTAD benchmark (73.6–90.99% image-level AUROC), with a larger BTAD pixel-AUROC/PRO gap flagged as an open question. Measured end-to-end (teacher plus student together, as deployed), the framework uses 1.93× fewer parameters (11.56 M vs. 22.35 M) and achieves 1.57× lower inference latency (4.436 ms vs. 6.954 ms) than a full-size STFPM-style baseline trained under an identical protocol. Under category-level paired testing (using 15 independent category means, the statistically appropriate unit given repeated measurements across shot counts), neither the image-level nor the pixel-level advantage over STFPM reaches statistical significance (two-sided p = 0.7197 and p = 0.188, respectively). Consequently, the framework’s established advantage over STFPM is its directly measured computational efficiency (1.93× fewer end-to-end parameters, 1.57× lower end-to-end latency), rather than accuracy. In contrast, against the RegAD-inspired baseline, both accuracy advantages remain highly significant under the same corrected test (two-sided p = 0.00201 for image-level and p = 0.00153 for pixel-level). A random, completely untrained student achieves only near-chance performance (52.7% image-level, 46.6% pixel-level AUROC), confirming that the observed localization signal reflects genuine distillation rather than an architectural artifact. A dedicated investigation of the framework’s weakest category (screw, near-chance performance) tested a rotation-sensitivity hypothesis directly across the full test set; the result was a significant correlation in the direction opposite to that hypothesized, and we report the mechanism as unresolved rather than settled. These results position the framework as a genuinely lightweight, empirically characterized — rather than definitively superior — approach to few-shot unsupervised industrial anomaly detection.
SN - 2045-2322
ER -