基于改进YOLOv5s+DeepSORT的煤块行为异常识别

Coal block abnormal behavior identification based on improved YOLOv5s + DeepSORT

煤块检测方法主要包括传统图像检测方法和深度学习目标检测方法。传统图像检测方法检测精度不高、实时性较差、无法对堆煤进行准确判断;深度学习目标检测方法虽然可以实现实时检测,但没有对煤块的数量、滞留和堵塞状态进行识别,而且识别模型参数较多。针对上述问题,提出了一种基于改进YOLOv5s+DeepSORT的煤块行为异常识别方法。首先通过摄像头和巡检机器人采集煤矿综采工作面带式输送机上煤块视频图像,并制作数据集。然后利用MobileNetV3_YOLOv5s_AF-FPN模型进行煤块图像目标检测:通过MobileNetV3替换原始YOLOv5s主干特征提取网络,减少参数量,提高推理速度;将YOLOv5s中原有的特征金字塔网络改进为增强特征金字塔网络(AF-FPN),以提高YOLOv5s网络对多尺度煤块目标的检测性能。利用DeepSORT进行煤块多目标跟踪:将改进YOLOv5s模型检测后的煤块图像作为DeepSORT的输入进行多目标跟踪,利用DeepSORT对煤块进行状态估计、数据关联匹配和跟踪器参数更新,确定跟踪结果,并对连续跟踪的煤块进行ID编码,对当前帧的煤块数量进行计数。最后在目标跟踪器中取出连续跟踪的目标,设置距离阈值,判断其是否滞留;设置数量阈值,判断其是否堵塞,最终实现煤块滞留和堵塞行为异常识别。利用自建dkm_data2021数据集对基于改进YOLOv5s+DeepSORT的煤块行为异常识别方法的可靠性进行实验验证,结果表明:改进YOLOv5s模型相比YOLOv5s模型平均检测精度提高了1.45%,参数量减少了35.3%,推理加速了12.7%,平均漏检率降低了11.08%,平均误检率降低了11.54%;基于改进YOLOv5s+DeepSORT的煤块行为异常识别方法检测精度为80.1%,可准确识别煤块滞留、堵塞状态,验证了该方法的可靠性。

Coal block detection methods mainly include traditional image detection methods and deep learning target detection methods. The traditional image detection method has low detection precision and poor real-time performance, and can not accurately determine the coal pile. Although the deep learning target detection method can achieve real-time detection, it does not identify the number, retention, and blockage of coal blocks.And there are many identification model parameters. To solve the above problems, a coal block abnormal behavior identification method based on improved YOLOv5s + DeepSORT is proposed. Firstly, video images of coal blocks on a belt conveyor in a fully mechanized coal mining face are collected by the camera and inspection robot, and data sets are made. Secondly, the MobileNetV3_YOLOv5s_AF-FPN model is used for detecting the coal image target. The original YOLOv5s backbone feature extraction network is replaced by MobileNetV3 to reduce the number of parameters and improve the reasoning speed. The original feature pyramid network in YOLOv5s is improved to AF-FPN to improve the detection performance of the YOLOv5s network for multi-scale coal targets. DeepSORT is used for multi-target tracking of coal blocks. The coal block image detected by the improved YOLOv5s is taken as the input of DeepSORT for multi-target tracking. DeepSORT is used to estimate the state of coal blocks, perform data association and matching, and update the tracker parameters to determine the tracking results. The continuously tracked coals are ID-coded, and the number of coals in the current frame is counted. Finally, the continuously tracked target is taken out from the target tracker, and a distance threshold is set. Whether the target is detained or not is determined. The quantity threshold is set to determine whether it is blocked. The identification of abnormal behavior of coal block retention and blocking state is finally realized. The reliability of the coal abnormal behavior identification method based on the improved YOLOv5s + DeepSORT is experimentally verified by using the self-built dkm_data2021 data set. The results show that compared with the YOLOv5s model, the average detection precision of the improved YOLOv5s model is improved by 1.45%, the parameter quantity is reduced by 35.3%, the reasoning is accelerated by 12.7%, the average missed detection rate is reduced by 11.08%, and the average false detection rate is reduced by 11.54%. The detection precision of coal block abnormal behavior identification method based on the improved YOLOv5s+DeepSORT is 80.1%, which can accurately identify the status of coal block retention and blockage. The result verifies the reliability of the method.