基于关节点遮挡推测的多相机手姿态估计方法

Hand Pose Estimation Using Multi-camera Based on Hand Joints Occlusion Prediction

基于视觉的手势交互技术被应用在航天员虚拟训练系统中,其中基于视觉的手姿态估计方法是关键。由于手部关节多、自由度高,因此航天员在操作过程中易出现手势自遮挡,此时难以准确重建航天员手势操作,进而难以构建合适的虚拟训练系统。当关节点严重自遮挡时,甚至出现不能被识别或姿态估计结果畸形的情况。为此提出了基于关节点遮挡预测的多相机融合手姿态估计方法,在不同视角利用单幅手部图像进行关节点遮挡预测和手姿态估计,再依据关节点遮挡预测结果,在决策端将各视角的手姿态估计结果进行融合;并采集双视角手势操作视频样例,设计融合实验进行验证。实验结果表明:通过融合可以有效解决单一视角自遮挡严重、手姿态难以估计的问题,避免了传统多相机方法的相机校准和数据特征匹配过程,在保证算法实时性的基础上提高了手姿态估计的精度。

Vision based gesture interaction has been applied in the astronaut virtual training system and the key is the hand pose estimation method. Due to the multiple joints and the high degree of freedom of the hand,the hand self-occlusion is easy to happen which makes it difficult to accurately rebuild the astronaut gesture operation or build a suitable virtual operating system. When the joints are severely self-occluded,it is unable to be identified or deformed estimation may occur. A novel method for hand pose estimation using multi-camera was proposed in this paper where hand pose form different views were fused directly depending on the joints occlusion estimation. The experimental results showed that this method could not only effectively solve the serious gesture self-occlusion problem,but also could avoid the tedious calibration process in traditional multi-camera methods. The result showed that the accuracy of hand pose estimation was improved and the problem of serious self-occlusion was well dealt with.