基于红外和可见光图像融合的铺丝缺陷检测方法

Detection method of defects in automatic fiber placement based on fusion of infrared and visible images

为提高纤维自动铺放(AFP)的质量,分析了目前视觉检测技术中单光谱检测技术成像的局限性,提出一种基于深度学习的红外与可见光联合检测缺陷手段以实现对铺放缺陷的检测、定位和分类。利用热红外图像与可见光图像中易检缺陷种类不同的特点,采用特征融合网络将两种光谱信息融合从而改善检测效果。考虑在线检测对实时性要求较高,为提高检测速度,采用单阶段检测网络作为检测框架,并依据纤维缺陷的长宽比分布严重不均的特点分析了单阶段检测网络中基于锚框方法的不足,提出采用无锚框的检测框架,增加改进的特征金字塔网络结构进行多尺度预测。以全类平均正确率(mAP)为衡量指标,实验结果相比单光谱检测方法、基于锚框检测方法和未使用改进特征金字塔结构分别提升了6.30%、6.64%和1.02%。通过Tensor RT加速后检测网络对每张608像素×608像素图像的检测时间小于20 ms,满足实时检测的需求,检测平均召回率超过88%,平均精确率超过82%,满足生产准确度需求。数据在试验台进行离线检测验证得出,并在大型龙门铺丝机上进行了在线测试。

To improve the quality of Automatic Fiber Placement(AFP),this paper analyzes the limitations of single spectrum detection technology in the field of visual inspection,and proposes a method based on deep learning for defect detection by fusion of infrared and visible images to realize detection,location and classification of fiber placement defects.According to the difference between the defects in thermal infrared image and visible image,the feature fusion network is used to fuse the two kinds of spectral information to improve the detection effect.To improve the detection speed,the single-stage detection network is used as the detection framework.We analyze the shortcomings of anchor-based method in the single-stage detection network according to the characteristics of seriously uneven distribution of length-width ratio of fiber defects,and propose a detection method with anchor-free network,in which an improved feature pyramid network structure is added for multi-scale prediction.Using mean Average Precision(mAP)as the measurement index,the experimental results improved by 6.30%,6.64%and 1.02%respectively compared with single spectrum detection method,anchor-based detection method and the detection method without improved feature pyramid structure.The detection time of each 608 pixels×608 pixels image is less than 20 ms after acceleration by Tensor RT,which meets the demand of real-time detection.The average recall of detection is more than 88%,and the average precision is more than 82%,which meets the demand of production accuracy.The data of this paper is verified by off-line detection on the test-bed,and is tested online on large gantry AFP equipment.