基于深度学习的多视图立体重建方法综述

Review of multi-view stereo reconstruction methods based on deep learning

多视图立体重建(Multi-view stereo Reconstruction,MVS Reconstruction)的目标是根据一组已知摄像机参数的多视角图像来重建场景的三维模型,是近年来三维重建的一类主流方法。本文针对最新的近百个基于深度学习的MVS方法做了较为系统的算法评估对比。首先,对现有的基于监督学习的MVS方法,按照特征提取、代价体构建、代价体正则化和深度回归的重建流程对各算法进行梳理,重点对代价体构建和正则化这两阶段的改进策略进行归纳总结,对于无监督的MVS方法,主要分析各算法损失项的设计,并按照其训练方式进行分类;其次,总结了MVS方法常用的实验数据集及其对应的性能评价指标,进一步研究特征金字塔结构、注意力机制、由粗到精等策略的引入对MVS网络性能的影响;此外,介绍了MVS方法的具体应用场景,包括数字孪生、自动驾驶、机器人技术、遗产保护、生物科学等领域;最后,提出关于MVS改进方向的建议,并对多视图三维重建未来的技术难点与研究方向进行探讨。

The goal of Multi-view stereo(MVS)Reconstruction is to reconstruct a 3D model of a scene based on a set of multi-view images with known camera parameters,which is a mainstream method of 3D reconstruction in recent years.This paper provides a algorithm evaluation comparison for the latest hun⁃dreds of MVS methods based on deep learning.First,we sorted out the existing supervised learningbased MVS methods according to the reconstruction process of feature extraction,cost volume construc⁃tion,cost volume regularization and depth regression,focusing on the summary of improvement strategies in the two stages of cost volume construction and cost volume regularization.For the unsupervised MVS methods,we mainly analyzed the design of the loss terms of each algorithm.It is classified according to its training mode.Secondly,we summarized the common datasets of MVS methods and their corresponding performance evaluation indexes,and further studied the introduction of strategies such as feature pyramid network,attention mechanism,coarse-to-fine strategy on the performance of MVS networks.In addition,it introduced the specific application scenarios of MVS methods,including digital twin,autonomous driv⁃ing,robotics,heritage conservation,bioscience and other fields.Finally,we made some suggestions for the improvement direction of MVS methods,and also discussed the future technical difficulties and the re⁃search directions of MVS 3D reconstruction.