面向移动增强现实的手势交互方法

The Method of Hand Gesture Interaction Based on Depth Learning and Hidden Markov Model

针对目前绝大多数移动设备无法感知深度信息,移动增强现实只能以触屏交互为主要手段这一问题,提出了一种基于深度学习及隐马尔可夫模型的手势交互方法(DL-HMM)。该方法利用深度传感器提取的骨骼特征和深度灰度图像,使用深度置信网络(DBN)和三维卷积神经网络(3DCNN)处理提取到的手势骨骼特征和深度图像特征融合成高维特征,采用隐马尔可夫模型的观测概率完成手势的训练和识别。实验结果表明本文提出的方法能有效提高手势识别率,提升了交互的准确性。

In view of the fact that the vast mobile devices can not sense the depth information at present,and mobile augmented reality can only be based on touch-screen interaction as the main method,this paper proposes a gesture interaction method based on depth learning and hidden Markov model（DL-HMM）.The method utilizes the depth sensor to extract the skeleton features and depth grayscale images.And the extracted gesture skeletal features and depth image features are processed using a depth-of-confidence network（DBN）and a 3 Dconvolutional neural network（3 DCNN）merged into high-dimensional features.The hidden Markov model observation probability is used to complete the gesture training and recognition.The experimental results show that this method can effectively improve the accuracy of gesture recognition and interaction.