采用DETR与先验知识融合的输电线路螺栓缺陷检测方法

Defect detection method of transmission line bolts based on DETR and prior knowledge fusion

为了解决深度学习模型无法学习螺栓目标的先验知识、仅通过视觉特征难以快速准确定位其缺陷以及螺栓缺陷样本数量有限、类别不平衡的问题,提出了将深度学习模型与螺栓先验知识相结合的方法。选取端到端目标检测(DETR)为基线模型,设计并实现了一种采用DETR与先验知识融合的改进DETR模型。首先,利用视觉-知识注意力模块将螺栓图像的视觉特征与螺栓先验知识有机融合,获得螺栓对应的增强视觉特征;然后,将增强视觉特征送入基于Transformer编码-解码结构的DETR模型框架中对螺栓目标进行识别与分类;最后,针对螺栓危急缺陷样本少及样本不平衡的问题,引入类增量学习损失函数(CILLF)来增强模型的鉴别能力,缓解螺栓缺陷样本间长尾分布问题。仿真实验结果表明:改进DETR模型在输电线路螺栓缺陷样本上的mAP相较于基线模型DETR提升了2.8个百分点;相较于主流Faster R-CNN,YOLOv5l模型,改进DETR模型在长尾分布下螺栓缺陷样本少的类别图像上的检测效果提升尤为显著。

In order to address the problem of deep learning model unable to learn the prior knowledge of bolt targets,difficulty in locating its defects quickly and accurately only through visual features,and the limited number and unbalanced categories of bolt defect samples,this paper proposed the method of incorporating the deep learning model and the prior knowledge of bolts.DETR(detection transformer)was selected as the baseline model,and an improved DETR model was designed and implemented by incorporating DETR and prior knowledge.First,the visual-knowledge attention module was used to fuse the visual features of the bolt image with the prior knowledge of the bolt,generating the enhanced visual features corresponding to the bolts.Then,the enhanced visual features were sent to the DETR model framework,which was based on the Transformer encoding-decoding structure,thus identifying and classifying bolt targets.Finally,to overcome the problem of few and unbalanced samples of bolt critical defects,a class incremental learning loss function(CILLF)was introduced to enhance the identification ability of the model and alleviate the long-tail distribution problem of bolt defect samples.The simulation results demonstrated that the improved DETR model achieved an increase of 2.8 percentage points in mAP on the transmission line bolt defect sample compared with the baseline model DETR.Compared with the mainstream Faster R-CNN and YOLOv5l models,the improved DETR model showed significant improvement in detecting category images with few bolt defect samples under the long-tail distribution.