基于深度特征融合的三维动态手势识别

3D dynamic gesture recognition based on depth feature fusion

在深度地图序列的手势识别中,针对不同的人在不同的时间或同一个人在不同的时间手势也不相同的问题,本文提出了特征加权融合和交叉主题测试法来进行基于深度地图序列的手势识别。首先,对于深度视频序列采用多级时间采样来生成含有相关手势信息的长、中和短3种不同长度的序列;其次,通过计算连续帧的绝对差提取时空信息生成深度运动图;然后,利用梯度方向直方图(histogram of oriented gradien,HOG)和局部二值模式(local binary patterns,LBP)从生成的深度运动图中提取形状和纹理特征,进行局部特征聚集描述符(vector of local aggregation descriptor,VLAD)编码;最后,采用主成分分析(principal component analysis,PCA)降维后将这两种特征进行加权融合和交叉主题测试后送到极限学习机器中进行分类识别。在公开具有挑战性的MSR Gesture 3D动态手势深度数据集上进行实验评估性能,所提的特征加权融合算法和交叉主题测试算法的识别率相较LBP和HOG算法融合的基础上分别提高0.82%和5.17%。实验结果表明,改进的方法具有更好的识别率。

In the gesture recognition based on depth map sequence,in order to solve the problems such as different people have different gestures at different time or the same person has different gesture at different times,weighted feature fusion and cross-subject test methods are proposed in this paper to perform the gesture recognition based on depth map sequence.Firstly,multilevel time sampling is used for depth video sequence to generate three different length(long,middle,short)sequences with relevant gesture information.Secondly,the spatiotemporal information is extracted by calculating the absolute difference of consecutive frames to generate a motion-in-depth map.Then,the histogram of oriented gradient(HOG)and local binary patterns(LBP)are used to extract shape and texture features from the generated motion-in-depth map,and the vector of local aggregation descriptor(VLAD)is coded.Finally,after principal component analysis(PCA)is used to reduce the dimension,these two features are sent to the extreme learning machine for classification and recognition after weighted fusion and cross-subject test.The performance is evaluated experimentally on the open and challenging dynamic MSR Gesture 3D depth data set.The recognition rate of the proposed weighted feature fusion algorithm and cross-subject test algorithm is respectively 0.82%and 5.17%higher than those on the basis of LBP and HOG algorithm fusions.Experimental results show that the improved method has better recognition rate.