基于深度神经网络的视觉手部姿态追踪系统

Vision Hand Attitude Tracking System Based on Depth Neural Network

提出一种新的图像手势识别系统。通过双目立体匹配从左右RGB图像中计算深度信息,结合深度神经网络推理三维手部姿态。由于二维跨越到三维的升维会缺失第三维度的数据,提出了三层结构的神经网络,通过学习已标注二维关键点的手部数据集,使其拥有分割二维图像中手部位置及识别关键点的能力,通过学习隐式的手部深度图数据集与合成手部模型数据集进行训练,将双目立体匹配计算出的深度信息与输入图像检测出的二维关键点结果输入到三维重建网络中,完成三维手部姿态的估计。

A new image gesture recognition system was presented.The depth information was calculated from the left and right RGB images by binocular stereo matching,and the 3 D hand posture was deduced by combining the depth neural network.Because the data of the third dimension will be lost when the two-dimensional data was upgraded to three-dimensional data,a three-layer structure of neural network was proposed.By learning the hand data set marked with 2 D key points,it had the ability to segment the position of hand and identify key points in 2 D image.By learning the implicit hand depth map data set and the synthetic hand model data set for training,the depth information calculated by binocular stereo matching and the two-dimensional key point results detected by the input image were input into the three-dimensional reconstruction network to complete the three-dimensional hand pose estimation.