基于注意力的共享参数胶囊网络

Attention-based capsule network with shared parameters

针对传统胶囊网络特征信息的传播冗余性和解构低效性问题,提出一种共享参数的注意力胶囊网络.该网络的优点主要体现于以下两方面:(1)提出注意力机制的动态路由方法,通过计算低级胶囊的相关性,使得在保留特征空间信息的同时更加关注相关性高的特征信息,并完成前向传播;(2)在动态路由层提出共享转换矩阵,基于低级胶囊投票一致性对高级胶囊激活,并通过共享转换矩阵减少模型的参数量,同时实现改进胶囊网络的稳健性.首先,通过5个公开数据集的分类对比实验,表明所提出胶囊网络在Fashion-MNIST、SVHN和CIFAR 10数据集上分别取得了5.17%、3.67%和9.35%的最好分类结果,而且在复杂数据集上具有显著的白盒对抗攻击鲁棒性;然后,通过在基于smallNORB和affNISH公开数据集的仿射变换对比实验,表明所提出的胶囊网络具有显著的仿射变换鲁棒性;最后,通过计算效率分析对比实验结果,表明所提出共享参数胶囊网络在不增加浮点运算的情况下,参数量比传统的胶囊网络减少4.9%,具有突出的计算量优势.

Aiming to handle the problem of propagation redundancy and deconstruction inefficiency of features in traditional capsule networks,this paper proposes an attention-based capsule network with shared parameters.The merits of such a network lie mainly in the following two issues:(1)A dynamic routing method based on an attention mechanism is proposed.This method calculates the correlation between low-level capsules to maintain the space information of features and pay more attention to the feature information with a high correlation,thus fulfilling the forward propagation;(2)A shared transformation matrix is developed in the dynamic routing layer.The high-level capsules are activated based on the voting consistency of the low-level capsules.Then,the transformation matrix with shared parameters is used to reduce the parameters of the model and obtain the robustness of the capsule network.Experimental results of comparison classification on five public datasets show that the proposed capsule network achieves the best classification results of 5.17%,3.67%and 9.35%on the Fashion-MNIST,SVHN and CIFAR 10 datasets,respectively.Moreover,it has significant robustness against the white-box anti-attack.In addition,the transformation experimental results on smallNORB and affNISH public datasets show that the proposed capsule network has obvious robustness to the transformation.Finally,the experimental results of computational efficiency show that the proposed capsule network with shared parameters reduces the parameters of traditional capsule networks by 4.9%without adding floating-point operations and has an overwhelming advantage in computation.