轨迹特征融合双流模型的动态手势识别

Dynamic Gesture Recognition Based on Trajectory Feature Recognition and Fusion Dual-flow Model

针对现有动态手势识别任务的识别率不高、鲁棒性不强等问题,提出一种新的动态手势识别方法。该方法将轨迹特征与手型时空特征融合到自适应分配权值的双流网络模型中,实现动态手势有效准确的识别。通过Kinect采集到整个动态手势的深度图序列和彩色图序列,从中提取出动态手势的轨迹特征曲线图与手型特征变化序列图;而后利用2D残差网络对动态手势的轨迹特征曲线图进行识别,得到轨迹信息识别结果;同时采用二模态训练后的3D双卷积神经网络对动态手势时空信息进行识别,得到时空网络识别结果;再根据两种网络的识别结果通过自适应分配权值进行融合得到最终的识别结果。实验结果表明,该方法在自制SKIG数据集上的识别率平均为99.52%,相比于其他方法取得了更高的识别精度,体现了该方法的鲁棒性与优越性。

Aiming at the problems of low recognition rate and low robustness of existing dynamic gesture recognition tasks,a new method for dynamic gesture recognition is proposed,which integrates trajectory features and hand-space-time features into a dual-flow network model with adaptively assigned weights to achieve effective and accurate recognition of dynamic gestures.The depth map sequence and color map sequence of the entire dynamic gesture are collected by Kinect,and the trajectory characteristic curve diagram and hand shape characteristic sequence diagram of the dynamic gesture are extracted from it.Then,the 2D residual network is used to identify the trajectory characteristic curve diagram of the dynamic gesture and obtain the trajectory information recognition result.At the same time,the 3D dual convolutional neural network after two-modal training is used to recognize the spatiotemporal information of dynamic gestures to obtain the recognition result of the spatiotemporal network,and then based on the recognition results of the two networks,adaptive weight assignment is performed.The final recognition result is obtained by fusion.The experiment shows that the average recognition rate of this method on the homemade SKIG dataset is 99.52%.Compared with other methods,it has achieved higher recognition accuracy,which shows its robustness and superiority.