融合静态手势特征和手部运动轨迹特征的手势交互方法

Gesture interaction method based on fusion of static gesture features and gesture track features

手势识别一直是人机交互研究的热点,由于受环境与视角的影响,单一特征不能很好地完成手势的识别,提出一种融合静态手势特征和手部运动轨迹特征的手势交互方法。该方法基于深度摄像头提取视觉信息,一方面获取RGB图像信息,对图像进行简单有效的预处理,将处理后的图像用卷积神经网络训练分类模型得到手部的静态手势特征;另一方面借助Kinect摄像头检测手部骨骼节点,利用基于向量角的轨迹特征识别方法得到手部的轨迹特征。将上述俩种特征融合,用静态手势特征和轨迹特征对动态手势进行描述,通过融合后的手势识别方法实现基于手势识别的人机交互。结果表明,融合静态手势特征和手部运动轨迹特征的手势交互方法比只使用一种手势特征的手势识别方法有较大提升。

Gesture recognition has always been a hot topic in human-computer interaction research. Due to the influence of environment and perspective, single feature can not complete gesture recognition well. This paper proposes a gesture interaction method which combines static gesture feature and gesture track feature. This method extracts visual information based on depth camera. On the one hand, it obtains RGB image information and preprocesses the image simply and effectively. The processed image is trained by convolution neural network to get the static gesture features of the hand. On the other hand, it detects the skeleton nodes of the hand with Kinect camera, and obtains the track features of the hand with the track feature recognition method based on vector angle The static gesture feature and track feature are used to describe the dynamic gesture, and the human-computer interaction based on gesture recognition is realized by the gesture recognition method after fusion. The results show that the proposed gesture interaction method based on the fusion of static gesture features and gesture track features is better than the gesture recognition method based on only one gesture feature.