基于半监督生成对抗网络的三维重建云工作室

Three-dimensional reconstruction cloud studio based on semi-supervised generative adversarial networks

由于固有的问题复杂性和计算复杂度,三维重建是计算机视觉研究和应用领域非常重要且富有挑战性的课题。目前已有的三维重建算法往往会导致重建的三维模型上存在着明显的空洞、扭曲失真或者模糊不清的部分,而基于机器学习的三维重建算法往往又只能重建简单的分离物体,并表示成三维体元形式。所以这些算法框架对于实际应用来说都还远远不够。从2014年起,生成对抗网络被广泛应用于半监督学习,以及产生非真实数据集的应用中。所以本文的重点是采用生产对抗网络原理,来获得高质量的三维重建效果。提出了一种新颖的半监督三维重建算法架构,命名为SS-GAN-3D。该算法通过训练生成对抗网络模型,使其达到收敛状态,以此来迭代式地提高原始三维重建模型的质量。SS-GAN-3D只需要将事先观测的二维图像作为弱监督样本,对于三维结构外形的先验知识或者参考观测都没有任何依赖。最终通过定性和定量实验,以及对实验结果的分析,该算法框架在Tanks&Temples和ETH3D标准三维重建测试集上,比目前最先进的三维重建方法有明显优势。基于SS-GAN-3D算法,又提出了三维重建云工作室解决方案。

Because of the intrinsic complexity in computation,three-dimensional(3 D)reconstruction is an essential and challenging topic in computer vision research and applications.The existing methods for 3 D reconstruction often produce holes,distortions and obscure parts in the reconstructed 3 D models.While the 3 D reconstruction algorithms based on machine learning can only reconstruct voxelized 3 D models for simple isolated objects,they are not adequate for real usage.From 2014,the generative adversarial network(GAN)is widely used in generating unreal dataset and semi-supervised learning.So the focus of this paper is to achieve high quality 3 D reconstruction performance by adopting GAN principle.A novel semi-supervised 3 D reconstruction framework,namely SS-GAN-3 D was proposed,which can iteratively improve any raw 3 D reconstruction models by training the GAN models to converge.This new model only takes 2 D observation images as the weak supervision,and doesn’t rely on prior knowledge of shape models or any referenced observations.Finally,through qualitative and quantitative experiments and analysis,this new method shows compelling advantages over the current state-of-the-art methods on Tanks&Temples and ETH3 D reconstruction benchmark datasets.Based on SS-GAN-3 D,the 3 D reconstruction studio solution was proposed.