基于卷积神经网络隐空间的虚拟对抗学习

Virtual adversarial learning based on latent space of convolutional neural network

对抗训练存在计算效率低的缺点,对此提出一种虚拟对抗学习的方法。在CIFAR-10和ImageNet(30)数据集上验证本方法,首先,建立阈值机制来挑选对抗源样本;然后,在对抗源样本的logits上添加扰动生成虚拟对抗样本,而非对抗源样本保持不变;最后,计算虚拟对抗样本和非对抗源样本的损失,通过反向传播更新网络权重。试验结果表明,与传统的对抗训练相比,本文方法在干净样本的测试精度上提升了大约7~14百分点,在扰动样本的测试精度上不亚于投影梯度下降(projected gradient descent,PGD)对抗训练的效果,尤其是在ImageNet(30)数据集上提升了4.62百分点。在训练效率上,与最慢的PGD对抗训练相比,本文方法的训练时间缩短了2/3左右。这些结果均证明了虚拟对抗学习既能提升对干净样本的预测精度,又能提高模型的鲁棒性;同时加快对抗训练过程,为对抗训练在工业环境的运用提供有效方法。

In response to the disadvantage of low computational efficiency of adversarial training,a virtual adversarial learning method was proposed to perform and verify on the CIFAR-10 and ImageNet(30)data sets.Firstly,a threshold mechanism was established to select adversarial source examples.Secondly,perturbations were added to the logits of adversarial source examples to generate virtual adversarial examples,while non-adversarial source examples remained unchanged.Finally,the losses of virtual adversarial examples and non-adversarial source examples were calculated,and the network weights were updated by back propagation.Experimental results show that compared with traditional adversarial training,this method improves the test accuracy of clean examples by about 7%to 14%,and is not inferior to PGD(projected gradient descent)adversarial training in the test accuracy of perturbed examples,especially on ImageNet(30)data set,which is increased by 4.62%.In terms of training efficiency,the training time of this method is shortened by about 2/3 compared with the slowest PGD adversarial training.These results demonstrate that virtual adversarial learning can not only improve the prediction accuracy of clean examples,but also enhance robustness of the model,and at the same time speed up the adversarial training process,which provides an effective reference for the landing of adversarial training in industrial environment.