基于改进YOLOv3的手势实时识别方法

Real-time Hand Gesture Recognition Method Based on Improved YOLOv3

针对基于人工建模方式的手势识别方法准确率低、速度慢的问题,提出一种基于改进YOLOv3的静态手势实时识别方法。采用卷积神经网络YOLOv3模型,将通过Kinect设备采集的IR、Registration of RGB、RGB和Depth图像代替常用的RGB图像作为数据集,并融合四类图像的识别结果以提高识别准确率。采用k-means聚类算法对YOLOv3中的初始候选框参数进行优化,从而加快识别速度。在此基础上,利用迁移学习的方法对基础特征提取器进行改进,以缩短模型的训练时间。实验结果表明,该方法对流式视频静态手势的平均识别准确率为99.8%,识别速度高达52 FPS,模型训练时间为12 h,与Faster R-CNN、SSD、YOLOv2等深度学习方法相比,其识别精度更高,识别速度更快。

The hand gesture recognition method based on artificial modeling has many problems such as low accuracy and slow speed.Therefore,this paper proposes a static hand gesture recognition method based on improved YOLOv3.By using the convolutional neural network YOLOv3 model,the commonly used RGB images are replaced by the IR,Registration of RGB,RGB and Depth images collected by Kinect equipment as dataset.The recognition results of these 4 types of images are fused to improve the recognition accuracy.The k-means clustering algorithm is used to optimize the initial candidate frame parameters in YOLOv3,so as to improve the recognition speed.On this basis,the transfer learning is used to improve the basic feature extractor to shorten the training time of the model.Experimental results show that for the recognition of static hand gestures in stream videos,the mean Average Precision(mAP)of the proposed method is 99.8% and the recognition speed is up to 52 FPS.The training time of the proposed model is 12 hours,and its recognition accuracy and speed is better than other deep learning methods such as Faster R-CNN,SSD and YOLOv2.