基于多监督的三维人体姿势与形状预测

Prediction of 3D Human Pose and Shape Based on Multi-Supervision

三维人体重建技术指通过图像或视频建立有相应姿势和体型的三维人体模型,其在虚拟现实(VR)、网游、虚拟试衣等方面有着十分广泛的应用前景.其中,参数化的三维人体模型由于参数数量的局限,重建精度较低,缺少细节特征.为了提升参数化三维人体模型的重建精度,增加其脸部与手部细节,提出一种基于多监督的三维人体模型重建方法.该方法结合传统的回归方法和优化方法,利用卷积神经网络回归出参数化人体模型的参数,得到一个较为粗糙的人体模型,将该模型作为初始模板进行拟合和迭代优化,将带有脸部和手部的全身密集关节点信息和轮廓信息作为回归网络的2D监督,同时使用迭代优化后的人体模型作为回归网络的3D监督,最终可由一幅图像获得一个多细节、高精度的参数化三维人体模型.定性分析结果表明,该方法为人体拟合过程提供正确的拟合方向,可有效减少非自然姿势的出现,提高三维人体模型重建的准确度.全身密集关节点监督可为模型增加更多手部与脸部的细节,而轮廓监督可减少重建的人体模型与图像中人体的像素级偏差.定量分析表明,该方法在数据集Human3.6M上的平均逐关节位置误差(MPJPE)为59.9 mm,较经典方法SPIN减少了4.16%,对关节点进行刚性对齐后模型的平均逐关节位置误差(MPJPE-PA)低至38.2 mm,较SPIN减少了7.06%.

The three-dimensional(3D)human reconstruction technology refers to the establishment of 3D human body model with corresponding pose and shape through images or videos,which has a wide application prospects in virtual reality,online games,virtual try-on,etc.Among them,the parametric 3D human body model has low reconstruction accuracy and lacks detailed features due to the limited number of parameters.To improve the reconstruction accuracy of the parametric 3D human body model and add details to the face and hands of the model,a novel human body model reconstruction method based on multi-supervision is proposed.This method combines the traditional regression method and optimization methods and uses a convolutional neural network to regress the parameters of a coarse parametric human body model,which is used as an initial template for fitting and iterative optimization.The dense joints of the whole body with the face and hands and silhouette information are used as a 2D supervision of the regression network,and the iteratively optimized model is used as a 3D supervision of the regression network.Finally,a multidetail and high-precision parametric 3D human-body model can be obtained from a single image.In the qualitative analyses,the proposed method provides a correct direction for the human body fitting process,which can reduce the appearance of unnatural poses and improve the accuracy of the reconstructed 3D human body model.The supervision of the whole-body dense joints can add more details to the face and hands of the human body model,while the silhouette information can reduce the pixel-level deviation of the reconstructed human body model from the human body in the image.Meanwhile,quantitative analyses show that the mean per joint position error(MPJPE)of the method on the Human3.6M dataset is 59.9 mm,which is 4.16% lower than that of the classical method skinned multiperson linear(SMPL)model,and the MPJPE after the Procrustes analysis is as low as 38.2 mm,which is 7.06% lower than that of SPIN.