基于多个关键点对应性的手部动态重建

Hand dynamic 3D reconstruction using multiple keypoint-to-keypoint correspondences

提出了一种新的基于单视图深度序列的手部运动跟踪和表面重建方法。在假设任意一对关键点的对应性在所有手部姿态上均一致基础上,使用一个平滑的手部网格模板来提供形状和拓扑先验,引入多个能量函数构造输入扫描与模板之间三维关键点到关键点的对应性,并将其整合到一个可用的非刚性配准管线中,以实现精确的表面拟合。通过最小化手部模板和输入深度图像序列之间的误差来捕获非刚性的手部运动。采用迭代求解的方法,通过显式的关键点到关键点之间的对应性,逐步细化手部关节区域的变形,从而达到快速收敛和合理变形的目的。在含有噪声的手部深度图像序列上的大量实验表明,该方法能够重建精确的手部运动,并且对较大的变形和遮挡具有鲁棒性。

We propose a new approach to robustly reconstruct non-rigid hand geometries and motions from single-view depth sequences.On the basis of the assumption that the correspondence between any pair of key points is consistent across all hand poses,a smooth hand mesh template is adopted to provide the shape and topology prior,and several energy functions are introduced to formulate these 3D keypoint-to-keypoint correspondences,which are then incorporated into the available non-rigid registration pipeline to achieve an accurate surface fit.The non-rigid hand motion is captured by minimizing the distortion error between a hand template and the partial input.An iterative solver is employed to gradually refine deformations around joint regions with explicit correspondences,leading to fast convergence and plausible deformations.Extensive experiments on hand depth sequences with noise demonstrate that our approach is capable of reconstructing accurate hand motions,and is robust for large deformations and occlusions.