融合粗粒度代价体及双边网格的轻量级多视图三维重建

Lightweight Multi-view Stereo Integrating Coarse Cost Volume and Bilateral Grid

针对基于深度学习的多视图立体(Multi-view Stereo,MVS)重建算法内存消耗过大、推理速度慢,以及对病态区域重建效果不佳的问题,提出了一种基于双边网格和融合代价体的轻量级级联的MVS重建网络。首先利用基于双边网格的代价体上采样模块将较低分辨率代价体高效地恢复成高分辨率代价体。随着采用轻量级的动态区域卷积和粗粒度代价体融合模块,提升网络对病态区域特征的表示能力以及对场景整体信息和结构信息的感知能力。实验结果表明,该网络在DTU数据集以及Tanks and Temples数据集上均取得了具有竞争性的结果,并且在内存消耗以及推理速度上都显著优于其他方法。

In order to tackle the problems of large memory consumption,poor real-time performance and poor reconstruction quality for low-textured areas of multi-view stereo reconstruction algorithm basedon deep learning,this paper proposes a lightweight cascade MVS reconstruction network based on bilateral grid and fused cost volume.Firstly,it builds the cost volume upsampling module based on learned bilateral grid,which can efficiently restore the low-resolution cost volume to the high-resolution cost volume.Then the dynamic region convolution and coarse cost volume fusion module are used to improve the network's ability to extract the feature of the challenging area and to perceive the global and structural information of the scene.Experimental results show that our method achieves competitive results on DTU dataset and tanks and temples benchmark,and is significantly better than other methods in memory consumption and inference speed.