基于正则化约束元学习优化器的深度学习模型

DEEP LEARNING MODEL BASED ON REGULARIZATION CONSTRAINED META LEARNING OPTIMIZERS

针对深度学习模型存在的过拟合和易受干扰等问题,在采用元学习优化器的基础上,研究并提出加入四种全新的正则化约束,用于训练元学习优化器。泛化能力和鲁棒性实验分别在两层和四层CNN的Mnist和Cifar10分类上进行,并与使用其他优化器的结果进行了比较,表明了加入正则化约束的元学习优化器,泛化能力得到提升,在FGSM和PGD攻击下的鲁棒性也得到了提升。在四种正则化约束中,Hessian矩阵的特征谱密度和迹作为正则化约束,泛化能力最好。

Aiming at the problems of overfitting and easy interference in the deep learning model,this paper studies and proposes to add four new regularization constraints to train the meta learning optimizers on the basis of using the meta learning optimizers.The generalization ability and robustness experiments were carried out on Mnist and Cifar10 classification of two layers and four layers CNN respectively.Compared with the results of other optimizers,it shows that the meta learning optimizer with regularization constraints improves the generalization ability and the robustness under FGSM and PGD attacks.Among the four regularization constraints,the characteristic spectral density and the trace of Hessian matrix are used as regularization constraints,and its generalization ability is the best.