基于生成对抗网络的人像修复

Portrait inpainting based on generative adversarial networks

人像修复广泛用于基于图像渲染和计算摄影的照片编辑。针对衣着的不同、高矮胖瘦的区别以及姿态的高自由度等因素给人像修复带来的困难,提出了一种基于生成对抗网络(GAN)的高效人像修复方法。算法分为两阶段:第一阶段基于编码器-解码器网络粗略修复图像,然后估计其中人体姿态信息;第二阶段基于姿态信息和GAN来精确修复人像。利用人像姿态信息来连接人像姿态关键点,形成姿态框架并执行膨胀操作,得到人像姿态掩码,以此构造人像姿态损失函数进行网络训练。实验结果表明,与Contextual Attention修复方法相比,所提方法的修复结果在结构相似度(SSIM)上提升了1%。该方法将人像姿态信息加入到修复过程中,有效地约束了待修复区域人像数据的解空间范围,加强了网络对人像姿态信息的关注程度。

Portrait inpainting was widely used in the photo editing based on image rendering and computational photography.A lot of factors including the variety in clothing,different body types such as tall,short,fat and thin size,the high freedom degree of human body pose,bring difficulties to portrait inpainting.Therefore,an efficient portrait inpainting method based on Generating Adversarial Network(GAN)was proposed.The algorithm consists two stages.During the first stage,the image was roughly inpainted based on an encoder-decoder network,and then the body pose information in the image was estimated.During the second stage,the portrait was accurately inpainted based on the pose information and GAN.Besides,the key points of the portrait pose were connected by using portrait pose information to form the pose framework and perform the dilation operation,and the portrait pose mask was obtained.Thereby,a portrait pose loss function was constructed for network training.The experimental results show that:compared with the Contextual Attention inpainting method,the proposed method has the SSIM(Structural SIMilarity index)increased by one percentage point.The method,by adding the portrait pose information into the portrait inpainting process,effectively constrains the solution space range of portrait data in the zone to be inpainted,and strengthens the network’s attention to the portrait pose information.