基于YOLOv4的目标检测知识蒸馏算法研究

Study on Knowledge Distillation of Target Detection Algorithm Based on YOLOv4

知识蒸馏作为一种基于教师-学生网络思想的训练方法,它通过复杂的教师网络来引导网络结构相对简单的学生网络进行训练,使得学生网络获得与教师网络相媲美的精度。知识蒸馏在自然语言处理和图像分类领域均有广泛的研究,而在目标检测领域的研究则相对较少且实验效果有待提升。目标检测的蒸馏算法主要是在特征提取层进行,而单一的特征提取层的蒸馏方式易导致学生不能充分学习教师网络知识,使得模型的精度较差。针对上述问题,通过在特征提取和目标分类与边框预测上都利用了教师网络的“知识”来指导学生网络进行训练,并提出了一种基于多尺度注意力机制的蒸馏算法,使得教师网络的知识更好地流向学生网络。实验分析表明,在YOLOv4基础上提出的蒸馏算法可有效地提高学生网络的检测精度。

Knowledge distillation,as a training method based on the teacher-student network,guides the relatively simple student network to be trained through the complex teacher network,so that the student network can obtain the same precision as the teacher network.It has been widely studied in the field of natural language processing and image classification,while the research in the field of object detection is relatively less,and the experimental effect needs to be improved.The Distillation Algorithm of object detection is mainly carried out in the feature extraction layer,and the distillation method of single feature extraction layer will cause students can’t learn the teacher’s network knowledge fully,which makes the accuracy of the model poorly.In view of the above problem,this paper uses the“knowledge”in feature extraction,target classification and border prediction of teacher network to guide student network to be trained,and proposes a multi-scale attention Distillation Algorithm to make the know-ledge of teacher network influence student network.Experimental results show that the distillation algorithm proposed in this paper based on YOLOv4 can effectively improve the detection accuracy of the original student network.