带式输送机上散状物料堆积视频实时检测

Video real-time detection of bulk material accumulation on belt conveyor

针对非接触式散状物料堆积检测方法存在检测速度慢、在图像模糊场景下检测精度低、深度学习模型内存需求大等问题,提出了一种基于轻量化Mask-RCNN(掩码-区域卷积神经网络)的带式输送机上散状物料堆积视频实时检测方法。首先,通过暗通道先验算法对采集的图像进行预处理,以减少运输装载过程中粉尘造成的图像雾化现象,提高图像边缘特征。针对传统的Mask-RCNN的主干网络ResNet无法满足在嵌入式平台上对散状物料堆积进行实时检测的需求问题,将去雾预处理后的图像输入到基于MobileNetV2+特征金字塔网络(FPN)的主干网络中进行特征提取,生成特征图,并对主干网络进行轻量化设计,以部署在嵌入式平台上,对实时采集图像数据进行实例分割。为更精确地找到分割物体的边缘,提出了在传统Mask-RCNN的掩码分支中添加边缘损失的方法,利用全卷积网络层生成掩码,结合Scharr算子构造边缘损失函数,融合目标分类、边界框回归、语义信息得到实例分割图像。最后,通过判断散状物料堆积掩码内的像素值是否超过预设阈值实现散状物料堆积检测。实验结果表明:所提方法的模型内存需求降低到以ResNet101为主干网络的模型的1/5,经图像去雾预处理后的平均精度均值提高了8%,单张图像平均检测时间为0.56 s,检测精度可达91.8%。

The non-contact bulk material accumulation detection method has problems, such as slow detection speed, low detection precision in image fuzzy scene, large memory requirement of deep learning model.In order to solve the above problems, a video real-time detection method of bulk material accumulation on belt conveyor based on lightweight Mask-RCNN(mask region-based convolutional neural network) is proposed.Firstly, the collected image is preprocessed by the dark channel prior algorithm to reduce the image fogging phenomenon caused by dust in the transportation and loading process and improve the image edge features. The traditional Mask-RCNN backbone network ResNet can not meet the requirement of real-time detection of bulk material accumulation on an embedded platform. In order to solve this problem, the defogging preprocessed image is input into the backbone network based on MobileNetV2 + feature pyramid network(FPN) for feature extraction. The feature graph is generated. The backbone network is designed to be lightweight. The backbone network is deployed on the embedded platform to collect image data in real-time for instance segmentation. In order to find the edge of the segmented object more accurately, a method of adding edge loss in the mask branch of traditional Mask-RCNN is proposed. The mask is generated by using full convolutional network layer. The edge loss function is constructed by combining the Scharr operator. The segmentation image is obtained by fusing object classification, bounding box regression and semantic information. Finally, the bulk material accumulation detection is realized by judging whether the pixel value in the bulk material accumulation mask exceeds a preset threshold value. The experimental results show that the memory requirement of the proposed method is reduced to1/5 of that of the model taking ResNet 101 as the backbone network. The average precision mean value after image defogging pre-processing is increased by 8%. The average detection time of one image is 0.56 s, the detection precision can reach 91.8%.