基于级联特征和图卷积的三维手部姿态估计算法

3D hand pose estimation algorithm based on cascaded features and graph convolution

针对手部的高自由度问题和结构相似问题引起的三维关键点姿态估计误差,本文提出了一套联合识别、检测以及姿态估计的三维手部骨架姿态回归网络。采用基于YOLOv3的预处理网络,提出基于级联多特征热度图的二维和三维关键点检测网络,并在特征提取网络架构中引入人体骨架手部约束,利用渐进的图卷积神经网络特征增强模块对骨架关键点结果进行进一步精细化修正,完成姿态由粗到细的调整。本文与现有多种算法在不同公共数据集下进行PCK指标和AUC指标比较,本文算法在不同测试集上的AUC指标均达到最高,平均AUC精度达到92.9%。实验表明本文方法可以通过单张二维数据准确、细致地估计三维手部姿态,并且在测试集与自然场景下均有较好表现。

For the 3D key point pose estimation error caused by the high degree of freedom problem and structural similarity problem of the hand,this paper proposes a novel 3D hand skeleton pose regression framework for joint identification,detection,and pose estimation.The framework firstly adopts a YOLOv3-based detector to obtain the position of hands,then a cascade pose estimation network is designed to get initial hand poses with 2D and 3D pose supervisions.Finally,considering the natural constrains in hand graph connection,we present progressive GCN module to further refine the initial hand pose from coarse to fine.This paper compares PCK metrics and AUC metrics with the state-of-the-art approaches under different public benchmarks,and the proposed method achieves the highest AUC metrics on different test sets,with an average AUC accuracy of 92.9%.The experiments illustrate that the proposed method is able to effectively and robustly predict 3D hand pose from monocular image,performing well in both test sets and in the wild.