基于注意力的多尺度蒸馏异常检测

Attention-based Multi-scale Distillation Anomaly Detection

基于知识蒸馏的异常检测方法中,教师网络远大于学生网络,使得所得特征表示在同一位置对应图像的感受野不同。为解决此问题,可使学生网络与教师网络结构相同。然而,学生与教师网络完全相同,使得在测试阶段,对于异常样本,教师网络与学生网络特征表示差异过小而影响异常检测的性能。为解决该问题,提出了基于高效通道注意力模块的多尺度知识蒸馏异常检测方法(ECA Based Multi-Scale Knowledge Distillation Anomaly Detection,ECA-MSKDAD),并结合数据增强操作提出了相对距离损失函数。使用经过预训练的网络作为教师网络,同时使用与教师网络结构相同的网络作为学生网络。在训练阶段,对训练样本采取数据增强操作以扩充训练集的规模,并在学生网络中引入高效通道注意力(Efficient Channel Attention,ECA)模块,以增加教师网络和学生网络之间的差异,增大异常数据的重构误差,进而提高模型的检测性能。此外,利用相对距离损失函数,将数据间关系从教师网络传递到学生网络,对学生网络的网络参数进行优化。在MVTec AD进行实验,与9种相关方法比较,所提方法在异常检测与异常定位上均取得更优的性能。

In the anomaly detection method based on knowledge distillation,the teacher network is much larger than the student network,so that the obtained feature representation has different visual fields corresponding to the image at the same position.In order to solve this problem,the structure of student network and teacher network can be the same.However,However,in the testing phase,the same student network and teacher network will lead to too small difference in their feature representation,which will affect the performance of anomaly detection.In order to solve this problem,ECA based multi-scale knowledge distillation anomaly detection(ECA-MSKDAD)is proposed,and a relative distance loss function is proposed based on data enhancement operation.The pre-trained network is used as the teacher network,and the network with the same network structure as the teacher network is used as the student network.In the training stage,the data enhancement operation is adopted for the training samples to expand the scale of the training set,and the efficient channel attention(ECA)module is introduced into the student network to increase the difference between the teacher network and the student network,increase the reconstruction error of the abnormal data and improve the detection performance of the model.In addition,the relative distance loss function is used to transfer the relationship between data from the teacher network to the student network,and the network parameters of the student network are optimized.Experiments on MVTec AD show that compared with nine related methods,the proposed method achieves better performance in anomaly detection and anomaly localization.