基于稠密点云的神经辐射场NeRF在视觉SLAM建图任务中的应用研究

Research on the application of NeRF based on dense point clouds in visual SLAM mapping tasks

基于点云等显式场景表达的传统SLAM技术在精度和鲁棒性上已经较为成熟,但在地图纹理和语义信息还原方面存在不足。为了提高SLAM技术在纹理和语义信息获取方面的性能,本文将具有可微渲染能力的神经辐射场(NeRF)引入到传统视觉SLAM系统中,提出了一种新型视觉SLAM方法DRM-SLAM。该方法使用ORB-SLAM3进行相机位姿估计,并结合关键帧的RGB信息和深度信息生成稠密点云,在动态体素网格的基础上,根据点云数据提供的三维几何信息在体素网格中进行采样减少NeRF调用多层感知机的频率。同时,该方法结合利用了多分辨率哈希编码和CUDA框架的NeRF实现,显著提升了NeRF的训练速度。在TUM、WHU-RSVI、Replica和STAR数据集上对本文提出的方法进行建图精度、完整度以及实时性测试的结果表明:DRM-SLAM利用稠密点云和NeRF体渲染技术填补了点云中的空洞,保留了传统的SLAM方法在位姿估计精度上的优势,提升了地图的纹理和材质的连续性。DRM-SLAM算法在Replica数据集上的帧率为22.3,该值远大于NICE-SLAM、iMap和Co-SLAM算法,证明了所提算法具有较高的实时性。在相同的场景下进行消融实验,基于稠密点云进行NeRF渲染比传统的NeRF的方法帧率提升了3倍,进一步证明了稠密点云可以加速NeRF收敛,充分展示了DRM-SLAM在地图重建方面的性能。

Traditional SLAM technologies based on explicit scene representations,such as point clouds,have matured in accuracy and robustness but fall short in capturing the texture and semantic information of the map.To address this limitation,this paper introduces neural radiance fields(NeRF)with differentiable rendering capabilities into the traditional visual SLAM system,proposing a novel visual SLAM method:DRM-SLAM(dense radiance mapper-SLAM).This method uses ORB-SLAM3 for camera pose estimation and combines the RGB and depth information of keyframes to generate dense point clouds.By utilizing a dynamic voxel grid,the method samples within the grid according to the three-dimensional geometric information provided by the point cloud data,thereby reducing the frequency of NeRF calling the multilayer perceptron(MLP).Additionally,the method incorporates multi-resolution hash coding and the CUDA framework′s NeRF implementation,significantly accelerating NeRF training speed.Tests on the TUM,WHU-RSVI,Replica,and STAR datasets demonstrate that DRM-SLAM effectively uses dense point clouds and NeRF volume rendering technology to fill gaps in point clouds,maintaining the pose estimation accuracy of traditional SLAM methods while enhancing texture and material continuity in the map.The DRM-SLAM algorithm achieves a frame rate of 22.3 on the Replica dataset,which is significantly higher than NICE-SLAM,iMap,and Co SLAM algorithms,showcasing its high real-time performance.Ablation experiments in the same scenario show that NeRF rendering based on dense point clouds increases the frame rate threefold compared to traditional NeRF methods,further proving that dense point clouds can accelerate NeRF convergence and demonstrating the effectiveness of DRM-SLAM in map reconstruction.