融合梯度和高斯过程回归的多视图重建方法

Multi-view reconstruction method integrating gradient and gaussian process regression

针对使用深度神经网络进行多视角图像三维重建时存在特征图对光照变化敏感以及重建不完整的问题,提出了一种融合梯度和高斯过程回归的多视图重建方法.首先,针对光照变化影响提取特征的问题,设计一个融合梯度的特征提取网络.通过对图像进行独立的梯度计算并在梯度与原图像的基础上使用卷积神经网络提取特征,提高了梯度信息在特征图中的彩响力,增强了特征图对光照变化因素影响的抑制力.其次,针对多视图重建中特征提取步骤只关注当前视图而没有考虑视图间的潜在空间关系的问题,提出一个融合高斯过程回归算法的视图特征增强模块,有效地增益了视图间相关信息对多视立体视觉重建任务的影响,提高了多视立体视觉重建结果的完整度.最后,通过衡量参考图像与相邻图像特征体之间的匹配程度计算不同视图对Costvolume的贡献度,重新构建符合视觉感知的CostVolume.在DTU和Tanks and Temples数据集上进行实验,结果表明,与主流的多视立体视觉重建方法相比,该方法在三维重建的完整度方面有较大提升,并且拥有良好的泛化性.

Aiming at the problems that the feature map is sensitive to illumination changes and the reconstruction is incomplete when using depth neural network for multi-view image 3D reconstruction,a multi view reconstruction method integrating gradient and Gaussian process regression is proposed,a multi-view reconstruction method integrating gradient and Gaussian process regression is proposed.Firstly,aiming at the problem that the illumination change affects the extraction of features,a feature extraction network integrating gradient is designed.Through the independent gradient calculation of the image and the convolution neural network is used to extract features based on the gradient and the original image,the influence of gradient information in the feature map is improved and the inhibition of the influence of illumination change factors is enhanced.Secondly,aiming at the problem that the feature extraction step in multi-view reconstruction only focuses on the current view without considering the potential spatial relationship between views,a view feature enhancement module integrating Gaussian process regression algorithm is proposed,which effectively increases the influence of relevant information between views on the multi-view stereo vision reconstruction task and improves the completeness of the multi-view stereo vision reconstruction results.Finally,the contribution of different views to CostVolume is calculated by measuring the degree of matching between the reference image and the adjacent image features,reconstructing CostVolume that conforms to visual perception.Experiments on the DTU and Tanks and Temples datasets show that compared with the mainstream multi-vision stereoscopic vision reconstruction method,the method has a great improvement in the completeness of three-dimensional reconstruction and has good generalization.