三维穿衣人体重建综述——从传统方法到高保真模型

A survey on 3D clothed human body reconstruction:from traditional methods to high-fidelity models

三维穿衣人体重建,在计算机图形学和三维视觉领域占有重要地位,广泛应用于多个方向。人体穿衣的多样性和动作的复杂性使得穿衣人体的高保真重建变得极其困难。深度学习技术优化了数据特征提取、隐式几何表示和神经渲染等关键环节,也推动了高保真穿衣人体重建技术的革命性进步。本文综述了人体重建的基本流程和组成模块,如各类输入数据、人体几何与动作表示、参数化模型以及三维到二维的渲染技术。同时,介绍了公开的穿衣人体数据集,简要回顾了近10年来人体重建算法的快速发展。本文详细探讨了几种主要的重建方法:稠密视角重建、非刚性运动重建(non-rigid structure from motion,NRSFM)、基于像素对齐的隐式几何重建以及生成模型方法。特别是,稠密视角重建能够生成高质量的人体几何,而NRSFM方法减少了对多视角的需求。基于像素对齐的方法重建细节丰富的人体几何,而生成模型方法利用多模态输入信息实现重建。最后总结了现有方法,并展望了未来研究方向,包括实现低成本高保真重建、加速重建过程和增强重建结果的可编辑性,以及在自然环境下进行重建的可能性。本文总结了近年来穿衣人体重建技术的进步,同时指出了未来研究可能集中的方向。

Three-dimensional human body reconstruction is a fundamental task in computer graphics and computer vision,with wide-ranging applications in virtual reality,human-computer interaction,motion analysis,and many other fields.This process is aimed at the accurate recovery of a three-dimensional model of the human body from given input data for fur⁃ther analysis and applications.However,high-fidelity reconstruction of clothed human bodies still presents difficulty given the diversity of human body shapes,variations in clothing,and complex human motion.Considerable progress has been attained in the field of three-dimensional human body reconstruction owing to the rapid development of deep learning meth⁃ods.In deep learning techniques,multilayer neural network models are leveraged for the effective extraction of features from input data and learning of discriminative representations.In human body reconstruction,deep learning methods achieved remarkable advancements through revolutionized data feature extraction,implicit geometric representation,and neural rendering.This article aims to provide a comprehensive and accessible overview of three-dimensional human body reconstruction and elucidate the underlying methodologies,techniques,and algorithms used in this complex process.The article introduces first the classical framework of human body reconstruction,which comprises several key modules that col⁃lectively contribute to the reconstruction pipeline.These modules encompass various types of input data,including images,videos,and three-dimensional scans,that serve as fundamental building blocks in the reconstruction process.The representation of human body geometry is a vital aspect of human body reconstruction.Capturing the nuanced contours and shapes that define the human form presents a challenge.The article also explores various techniques for geometric represen⁃tation,from mesh-based approaches to implicit representations and voxel grids.These techniques capture intricate details of the human body while ensuring that body shapes and poses remain realistic.The article also delves into the challenges associated with the reconstruction of clothed human bodies and examines the efficacy of parametric models in encapsulating the complexities of clothing deformations.Representation of human body motion is another crucial component of human body reconstruction.Realistic reconstructions require accurate modeling and capture of the dynamic nature of human move⁃ments.The article comprehensively explores various approaches to modeling human body motions,including articulated and non-rigid ones.Techniques,such as skeletal animation,motion capture,and spatiotemporal analysis,are discussed for the accurate and lifelike representations of human body motion.Parametric models also contribute to human body recon⁃struction because they provide a concise and expressive representation of the complete human body.The article further examines optimization-based methods,regression-based approaches,and popular parametric models,such as skinned multi-person linear(SMPL)and SMPL plus offsets,for human body reconstruction.These models allow the capture of real⁃istic body shapes,poses,and clothing deformations.The article also discusses the advantages and limitations of these mod⁃els and their applications in various domains.Deep learning techniques have had a transformative influence on threedimensional human body reconstruction.The article explores the application of deep learning methodologies in data feature extraction,implicit geometric representation,and neural rendering and highlights the advancements achieved in leveraging convolutional neural networks,recurrent neural networks,and generative adversarial networks for various aspects of the reconstruction pipeline.These deep learning techniques considerably improve the accuracy and realism of reconstructed human bodies.Furthermore,publicly available datasets have been specifically curated for clothed human body reconstruc⁃tion.These datasets serve as invaluable resources for benchmarking and evaluation of the performance of various recon⁃struction algorithms and enable researchers to compare and analyze the effectiveness of different techniques to foster advancements in the field.Then,a comprehensive survey of the rapid advancements in human body reconstruction algo⁃rithms over the past decade is presented.The survey highlights breakthroughs in dense view reconstruction,non-rigid structure from motion(NRSFM)methods,pixel-aligned implicit geometry reconstruction,generative models,and param⁃eterized models.The discussion is also focused on the strengths,limitations,and potential applications of each approach to provide readers with holistic insights into the current state-of-the-art techniques.In conclusion,this article offers an indepth and accessible exploration of three-dimensional human body reconstruction and covers a wide range of topics,such as data acquisition,geometry representation,motion modeling,and rendering of modules.The article not only summarizes existing methods but also provides insights into future research directions,such as the pursuit of high-fidelity reconstruc⁃tions at reduced costs,accelerated reconstruction speeds,editable reconstruction outcomes,and the capability to recon⁃struct human bodies in natural environments.These research endeavors increase the accuracy,realism,and practicality of three-dimensional human body reconstruction systems and unlock new possibilities for various applications in the academia and industry.