基于多任务学习的有限样本多视角三维形状识别算法

3D Shape Recognition Based on Multi-task Learning with Limited Multi-view Data

随着三维扫描技术的快速发展,三维形状分析得到了学术界的广泛关注;尤其是深度学习在计算机视觉上取得的显著成功,使得基于多视图的三维形状识别方法成为了目前三维模型识别的主流方式。已有研究表明,三维数据集的数量对于最终的分类精度是一个非常重要的影响条件。然而,由于专业三维扫描设备的限制,三维形状数据难以采集。实际上,现有的公共基准三维数据集的规模远远小于二维数据集,三维形状分析的发展因此受到阻碍。为了解决这一问题,文中主要研究在极小数据样本情况下,三维形状识别问题的优化解策略。受多任务学习的启发,搭建了多分支的网络结构,并引入基于度量学习的辅助比较模块,用于挖掘类内和类间的相似性和差异性信息。网络模型包括主支路与辅助支路,分别使用不同的损失函数对应不同的训练任务,并使用权值超参数平衡多项损失。主支路获得预测分类,使用交叉熵损失函数进行更新;辅助支路得到不同样本间的相似性得分,使用均方差损失函数进行更新。为保证特征向量被投影到同一个空间中,主、辅助支路共享相同的特征提取模块,在训练阶段共同更新参数,在测试阶段仅使用主支路获得的分类结果。在两个公开的三维形状基准数据集上的大量实验结果表明,所提网络结构与训练策略相比传统方法,在少样本的情况下可以显著提高特征模块对不同类别的区分能力,获得更优的识别结果。

With the rapid development of 3D scanning technology,3D shape analysis has been widely concerned by researchers.Especially with the significant success of deep learning in computer vision,the approaches of 3D shape recognition based on multi-view have become the dominant methods.In the previous work,we notice that the amount of 3D shapes is essential for the recognition accuracy.However,due to the limitation of professional 3D scanning equipment,the 3D shape data is hard to collect.In fact,the scale of existing benchmark datasets is far smaller than that of 2D datasets which impedes the development of 3D shape analysis.In order to solve this problem,we mainly develop an optimal strategy of 3D shape recognition with limited data.Inspired by multi-task learning,we develop a novel network with multiple branches and construct an auxiliary comparison module based on metric learning to exploit the similarity and discrepancy between different samples intra-class and inter-class.The proposed network mainly includes a primary branch and an auxiliary branch,which respectively use disparate loss functions with different training tasks and hyper-parameter to balance different loss items.The primary branch aims to obtain the prediction of classification and uses Cross Entropy Loss function to train it.While the similarity scores of different samples are calculated by the auxiliary module,and the Mean Square Error is used to update this branch.Both two branches share the same feature extractor to project all samples into the same representation space and train the structure jointly in training phase,while the primary branch would be used in testing phrase to calculate the accuracy.Extensive experimental results have reported on two public 3D shape benchmark datasets which demonstrate the effectiveness of our proposed architecture to enhance the discriminative power and achieve better performance compared with traditional methods,especially in the situation where merely has limited multi-view data.