路径聚合的深度学习多视角立体三维重建算法

Deep learning multi-view stereo 3D reconstruction algorithm based on path aggregation network

针对当前多视角立体视觉方法在低纹理、重复纹理等复杂区域上的重建完整度低的问题,提出一种基于路径聚合网络的多视角立体视觉方法 PathMVSNet。PathMVSNet在常规的特征金字塔网络后增加一条自底向上的路径聚合结构,强化底层定位特征在网络中的传递,并将多尺度特征图经过可变形卷积层和卷积注意力机制模块增强特征;采用级联的代价体构建方式,由粗到细的进行深度预测;多视角特征体通过可学习的自适应权重网络对特征体进行加权聚合得到代价体。PathMVSNet在DTU数据集上进行训练和评估,与CasMVSNet进行比较实验,在平均完整度误差(Comp)、平均准确度误差(Acc)、平均整体性误差(Overall)上分别降低了9.3%、7.9%、8.6%。通过消融实验证明,PathMVSNet可以有效提升重建模型的完整度和整体质量。

In view of the low level of reconstruction completeness in the complex areas with low texture and repeated texture,an MVS(multi-view stereo)method PathMVSNet on the basis of path aggregation network(PANet)is proposed.After the conventional feature pyramid network,a bottom-up path aggregation structure PathMVSNet is incorporated to reinforce the transmit of underlying localization features in the network.In addition,the multi-scale feature map is subjected to feature enhancement by deformable convolutional layers and convolutional block attention module(CBAM).A cascading cost volume construction is adopted for depth prediction from coarse to fine.Additionally,the multi-view feature volume is weighted and aggregated by a learnable adaptive weighting network to obtain the cost volume.The PathMVSNet is trained and evaluated on the dataset DTU.Experiments are operated to make comparative analysis between the CasMVSNet and the PathMVSNet.The average completeness error,average accuracy error and average overall error of the PathMVSNet are reduced by 9.3%,7.9%and 8.6%,respectively.The ablation experiments prove that the PathMVSNet can enhance the level of completeness and overall quality of the reconstruction model effectively.