基于双通道Mask-RCNN的手势识别

Gesture Recognition Based on Dual Channel Mask RCNN

光照、复杂的背景以及多变的手势一直以来都是手势识别的难点,随着采集设备的不断发展,深度相机采集到的深度数据为彩色数据增添了更多的特征信息,有效地解决了光照变化带来的手势识别率不高的问题。此外,深度学习能够从图像中自动学习有效的手势特征,避免了传统方法在特征提取时遇到的耗时较多的问题。因此结合深度学习与RGB-D数据的优势,提出一种以RGB-D数据为输入的双通道的Mask RCNN网络。该网络在原有网络的基础上增加了一个深度特征提取通道,并在特征层面上将RGB特征和预处理后得到的深度特征进行融合。最后为了避免训练模型出现过拟合现象,提出采用扰动交叠率算法,进一步提高手势检测的识别率。实验结果表明,本方法相比较于仅采用彩色图的Mask RCNN网络,在简单数据集上的识别率提高了2.77%,在困难数据集上提高了4.39%。

Abstrat:Lighting,complex background and changeable gestures are always the difficulties of gesture recognition.With the continuous development of acquisition equipment,the depth data collected by the depth camera adds more characteristic information to the color data,which effectively solves the problem of low recognition rate of gestures caused by lighting changes.Therefore,in this paper,the advantages of deep learning and RGB-D data are combined,and a dual-channel Mask RCNN network is proposed that takes RGB-D data as input.A deep feature extraction channel is added on the basis of the original network;and the RGB features are fused with the depth features obtained after preprocessing on the feature level.Finally,in order to avoid overfitting of the training model,a perturbation overlap rate algorithm is proposed to further improve the recognition rate of gesture detection.The experimental results show that compared with the Mask RCNN network using only color maps,the method in this paper improves the recognition rate on simple data sets by 2.77%and increases on difficult data sets by 4.39%.