Traditional neural radiance fields for rendering novel views require intensive input images and pre-scene optimization,which limits their practical applications.We propose a generalization method to infer scenes from ...Traditional neural radiance fields for rendering novel views require intensive input images and pre-scene optimization,which limits their practical applications.We propose a generalization method to infer scenes from input images and perform high-quality rendering without pre-scene optimization named SG-NeRF(Sparse-Input Generalized Neural Radiance Fields).Firstly,we construct an improved multi-view stereo structure based on the convolutional attention and multi-level fusion mechanism to obtain the geometric features and appearance features of the scene from the sparse input images,and then these features are aggregated by multi-head attention as the input of the neural radiance fields.This strategy of utilizing neural radiance fields to decode scene features instead of mapping positions and orientations enables our method to perform cross-scene training as well as inference,thus enabling neural radiance fields to generalize for novel view synthesis on unseen scenes.We tested the generalization ability on DTU dataset,and our PSNR(peak signal-to-noise ratio)improved by 3.14 compared with the baseline method under the same input conditions.In addition,if the scene has dense input views available,the average PSNR can be improved by 1.04 through further refinement training in a short time,and a higher quality rendering effect can be obtained.展开更多
针对目前在混合现实(MR)环境中高效率建立高质量三维(3D)模型的需求,基于神经辐射场算法(NeRF)的三维重建技术,提出了一种基于Laplacian算子的数据集优化算法。首先,围绕某线切割设备录制了一段1 min 51 s的视频,并采取等距提取视频帧...针对目前在混合现实(MR)环境中高效率建立高质量三维(3D)模型的需求,基于神经辐射场算法(NeRF)的三维重建技术,提出了一种基于Laplacian算子的数据集优化算法。首先,围绕某线切割设备录制了一段1 min 51 s的视频,并采取等距提取视频帧的方式,获取了训练数据集;然后,使用Laplacian算子对数据集进行了优化,同时保留了原始数据集作为对比,使用了基于NeRF算法的重建方式与传统的基于COLMAP的稠密点云重建方式,分别对两组数据集进行了三维重建;最后,在重建精度与重建速度方面,对不同重建方式、不同重建数据集的重建结果进行了比较。研究结果表明:COLMAP稠密点云重建耗时是基于NeRF重建耗时的9.98倍,而相较于COLMAP稠密点云重建,使用NeRF重建方式的模型表面缺陷较少;此外,使用Laplacian算子优化的数据集的NeRF重建在峰值信噪比(PSNR)和结构相似性(SSIM)指标上分别提升了2.43%、0.72%,有利于提升重建模型的质量。研究结果支持混合现实技术在制造业数字化转型中的应用,可为其提供有益的参考。展开更多
基金supported by the Zhengzhou Collaborative Innovation Major Project under Grant No.20XTZX06013the Henan Provincial Key Scientific Research Project of China under Grant No.22A520042。
文摘Traditional neural radiance fields for rendering novel views require intensive input images and pre-scene optimization,which limits their practical applications.We propose a generalization method to infer scenes from input images and perform high-quality rendering without pre-scene optimization named SG-NeRF(Sparse-Input Generalized Neural Radiance Fields).Firstly,we construct an improved multi-view stereo structure based on the convolutional attention and multi-level fusion mechanism to obtain the geometric features and appearance features of the scene from the sparse input images,and then these features are aggregated by multi-head attention as the input of the neural radiance fields.This strategy of utilizing neural radiance fields to decode scene features instead of mapping positions and orientations enables our method to perform cross-scene training as well as inference,thus enabling neural radiance fields to generalize for novel view synthesis on unseen scenes.We tested the generalization ability on DTU dataset,and our PSNR(peak signal-to-noise ratio)improved by 3.14 compared with the baseline method under the same input conditions.In addition,if the scene has dense input views available,the average PSNR can be improved by 1.04 through further refinement training in a short time,and a higher quality rendering effect can be obtained.
文摘针对目前在混合现实(MR)环境中高效率建立高质量三维(3D)模型的需求,基于神经辐射场算法(NeRF)的三维重建技术,提出了一种基于Laplacian算子的数据集优化算法。首先,围绕某线切割设备录制了一段1 min 51 s的视频,并采取等距提取视频帧的方式,获取了训练数据集;然后,使用Laplacian算子对数据集进行了优化,同时保留了原始数据集作为对比,使用了基于NeRF算法的重建方式与传统的基于COLMAP的稠密点云重建方式,分别对两组数据集进行了三维重建;最后,在重建精度与重建速度方面,对不同重建方式、不同重建数据集的重建结果进行了比较。研究结果表明:COLMAP稠密点云重建耗时是基于NeRF重建耗时的9.98倍,而相较于COLMAP稠密点云重建,使用NeRF重建方式的模型表面缺陷较少;此外,使用Laplacian算子优化的数据集的NeRF重建在峰值信噪比(PSNR)和结构相似性(SSIM)指标上分别提升了2.43%、0.72%,有利于提升重建模型的质量。研究结果支持混合现实技术在制造业数字化转型中的应用,可为其提供有益的参考。
