二维碰撞感知的单张图像参数化人体与服装重构

2D Collision-Aware Human Body and Garments Reconstruction Based on Parametric Models from a Single Image

已有的基于图像的人体与服装重构大都不考虑身体与衣物之间的交互,导致独立重构结果互相穿刺.本文提出一个二维碰撞感知的从单张图像重构人体与服装的优化方法,利用SMPL和TailorNet参数化模型来分别表示人体和衣服的三维形状,进而建立约束能量求解人体形状参数,运动参数和服装细节控制参数.我们的方法在初始化阶段对输入图像进行了语义分割以及二维关节点的估计,并采用human mesh recovery初步估计人体的形状与运动参数作为能量优化的初始值.我们的能量优化由两部分构成:其一是形状与姿态约束,利用图像中人体的关节位置和着装人体区域对三维参数化模型的投影的关节位置和投影区域进行约束,保证重建模型与图像在形状与姿态上的一致性;其二是人体与衣服的碰撞约束,引入重构人体与服装模型的二维投影区域间的误差对人体与衣服进行碰撞约束,以避免相互穿刺,考虑到基于投影的约束对视点敏感,我们在三维空间中进行视点采样,从而建立多视角的二维投影约束.考虑到能量中包含了TailorNet,不容易计算梯度,我们利用爬山法交替地对人体形状,姿态和服装尺寸参数进行优化求取最优解.最后,通过一系列实验对本文方法和最近的一些相关工作进行了定性和定量分析,结果表明本文方法能有效缓解人体与服装穿刺,重构精度也更高。

With the rapid development of multimedia and digital technology,people now are able to obtain a large number of high-quality images or photos easily with the newest devices like digital camera or smartphone.Dressed people are always the focus in images or photos,and people are desired to acquire data with higher reality than two-dimensional data such as image.As a conse-quence,numerous methods attempt to reconstruct human body and outer garments based on a single image or photo.However,plenty of recent image-based methods reconstruct human body and garments separately without considering the interaction between them,which leads to mutual interpenetration.This paper purposes a method to reconstruct human body as well as outer garments from a single image based on 2D collision detection.It separately utilizes SMPL and TailorNet as body and garment parametric models,and then establishes an energy to jointly optimize the shape and pose parameters of the human model as well as the style parameters of the garment models.Our method starts with an preprocessing stage towards the input image which includes semantic segmentation of all garments and 2D joints estimation of the human body.After preprocessing stage,we employ human mesh recovery with the input image to estimate the shape and pose parameters of the human body as the initial values of the optimization.Although human mesh recovery is able to estimate the shape and pose parameters of SMPL based on the input image,the reconstructed 3D human body suffers the inaccurate shape and pose because of the ambiguity between the human body and the garments in the image.Hence,it is necessary to conduct a further optimization to estimate the shape and pose parameters of human body as well as the unknown style parameters of the outer garments with higher accuracy.Our optimization energy consists of two parts:The first is the shape and pose constraint,which penalizes the difference of the 2D joint positions and the region of dressed person between the image and the projection of the 3D parametric models;The other is a collision constraint between human body and garments,which introduces an error measurement of 2D projection areas between human body and outer garments to prevent interpenetration;In addition,considering that projection-based constraint is sensitive to viewpoints,we take a further step to sample more viewpoints to project the 3D models onto 2D spaces so as to reinforce the 2D collision constraint.All constraints mentioned above are unified into one optimization framework in order to take all factors into account.Considering the difficulty of computing the gradients of TailorNet due to the complicated network structure,we employ a global optimization algorithm named the hill-climbing algorithm to alternatively optimize shape,pose and style parameters in the energy.In the end,we conduct a variety of experiments to compare our results with those of state-of-the-art methods in both qualitative and quantitative analysis,which shows that the proposed approach can effectively alleviates penetration between body and garments,and achieve higher accuracy.