插值Loop细分Pytorch3D三维VR重建

Pytorch3D 3D VR reconstruction based on interpolation Loop subdivision

针对Pytorch3D三维重建的模型易产生空洞、断角和狭长三角形的问题,提出一种基于插值Loop细分的Pytorch3D三维重建方法。首先从20个视角对1个包含纹理的三维模型进行采样,采样后的图像合成为原始数据集;接着使用ResNet-50网络学习图像特征,使用三维卷积模块提取三维特征;随后用插值Loop细分原始网格,并以结果网格作为变形网格,将提取的特征投影到网格顶点上,利用图卷积神经网络实现变形,最终完成三维重建。结果表明,所提方法较好地解决了原版Pytorch3D方法的空洞、断角、狭长三角形问题,重建损失值和Hausdorff距离显著下降,三维重建可视化效果更好。

To address the problem related to the fact that the 3D reconstruction model of Pytorch3D is prone to produce holes,broken corners,and slender triangles,a Pytorch3D 3D reconstruction method based on Interpolation Loop subdivision is proposed.This method employs a 3 d model with texture firstly,and samples images from the target model from 20 perspectives and positions to synthesize data sets.Subsequently,ResNet-50 network is used to learn image features and the 3D convolution module is used to extract 3D features.Then the mesh subdivided by the Interpolation Loop is used as the deformation mesh.And the extracted features are projected onto the mesh vertices.Finally,the deformation is realized by the mesh using the graph convolutional neural network.And it completes the 3D reconstruction.Experimental results show that the proposed method can reconstruct target model well.And the reconstructed loss value and the Hausdorff distance are significantly improved.Therefore,the mesh reconstructed by our method is closer to the target model,and its visualization effect is better.