基于神经辐射场结构模糊优化的三维重建

3D reconstruction based on neural radiance field structure ambiguity optimization

少视图三维重建仅需要较少的视图来恢复物体的三维几何形状或场景。但由于少视图存在不同视角上的覆盖不足、缺乏足够的信息来准确还原三维场景的问题,会导致重建结果的不准确或者模糊,尤其针对具有复杂几何结构的场景难以捕获到场景的结构信息。本文提出了一种基于神经辐射场的框架,利用准确成本代价体关联前景和后景深度信息解决结构模糊的问题。首先,利用金字塔网络提取前景和后景的局部特征,加强对场景细节的捕捉,引入了Self-attention机制,以确保在特征提取过程中关注关键区域。然后,通过自适应感受野模块实现特征尺度的平滑传递,以此来构建一个准确的特征代价体。最后,引入随机结构相似性损失,利用局部区域像素作为整体监督替代像素逐点监督,以更全面地捕捉场景中的结构信息。在DTU数据集、LLFF数据集以及NeRF数据集上的PSNR分别提升了0.48 Db、0.1 Db、0.47 Db,实验数据表明,本文方法能有效解决少视图三维重建中信息不足导致的结构模糊问题。

Few view 3D reconstruction requires only fewer views to recover the 3D geometry of an object or scene.However,the lack of sufficient information to accurately restore the 3D scene due to the undercoverage on different viewpoints in the fewer views can lead to inaccurate or blurred reconstruction results.In this paper,a neural radiation field-based framework is proposed to solve the problem of structural blurring by utilizing accurate cost costumers to correlate foreground and back view depth information.First,the local features of the foreground and the backscene are extracted using a pyramid network to enhance the capture of the scene details,and the self-attention mechanism is introduced to ensure that the key regions are attended to during the feature extraction process.Then,the smooth transfer of feature scales is realized by the adaptive sensory field module as a way to construct an accurate feature cost volume.Finally,random structural similarity loss is introduced to replace pixel-by-pixel supervision by utilizing local area pixels as a whole supervision to capture the structural information in the scene more comprehensively.The experimental results show that in the DTU dataset,PSNR and LPIPS can achieve optimal results and SSIM can achieve suboptimal results compared with the comparison methods,and PSNR,LPIPS and SSIM are improved by 0.478,0.001 and 0.01,respectively.Compared with the baseline model,experiments on the DTU dataset,the LLFF dataset and the NeRF dataset show that the method proposed in this paper can effectively solve the structural ambiguity problem caused by insufficient information in view less 3D reconstruction.