类dropout的具有新型栈式结构的层次支持向量机

DROPOUT-LIKE HIERARCHICAL SUPPORT VECTOR MACHINE WITH NOVEL STACKED STRUCTURE

基于对抗学习提出一种类dropout的具有新型栈式结构的层次支持向量机(D-S-SVM)。随机抽取一定比例的样本攻击其标签类型使其成为对抗样本,利用支持向量机对包含对抗样本的训练集进行对抗学习生成对抗支持向量机(A-SVM)。通过栈式结构原理逐层级联一定数量的子分类器(即A-SVM)构建D-S-SVM。在该模型中计算子分类器输出误差对输入样本的一阶梯度信息,并结合dropout将部分一阶梯度信息嵌入到原输入样本特征中生成新样本作为下一个子分类器的输入。该模型不仅提供了一种新颖的层次结构级联方式,且实验结果表明它能够逐层提高数据分类精度且具有较强的泛化性能。

A dropout-like hierarchical support vector machine(SVM)with novel stacked structure(D-S-SVM)based on adversarial learning is proposed in this paper.The adversarial attacks were carried out on a certain percentage of training samples,which were randomly selected from the given training dataset,and the adversarial samples can be obtained.By training the support vector machine(SVM)on the training dataset including the adversarial samples,the resultant adversarial support vector machine(A-SVM)could be generated.Take the A-SVMs as sub-classifiers in a deep learning model,the D-S-SVM could be constructed by stacking a certain number of sub-classifiers based on the stacked generalization principle.In the D-S-SVM,the first-order gradient information corresponding to the output error of the sub-classifier in current layer with respect to the features of all inputs were calculated,and then they were integrated with the ideology of the dropout.The inputs were updated by embedding the resultant first-order gradient information such that the updated inputs could be taken as the inputs of the sub-classifier in next layer.The D-S-SVM provides a novel stacked way and extensive experimental results demonstrate that it can improve the classification precision in a layer-by-layer manner and has better generalization performance.