用双层减样法优化大规模SVM垃圾标签检测模型

Double-layer reduction method optimizes large scale SVM social spam detection model

针对支持向量机在训练大规模数据集时出现的速度瓶颈问题,提出一种新的减样方法,称为双层减样法。数据减样时,双层减样法从粗、细粒度两个层次削减样本。粗粒度约减时,利用核空间距离聚类法,以簇为单位削减冗余子集;细粒度约减时,以点为单位挑选剩余点集中的支持向量。实验表明,双层减样法能有效地压缩样本数据,同时还能放大数据集的分类特征,提高分类器的分类精度。将此法应用于大规模SVM垃圾标签检测模型的训练集优化上,能明显提高检测模型的训练速度。双层减样法将粒度和层次的概念引入减样法中,在约减时适时改变约减幅度,这比传统减样法更具有优势。

In order to improve the low efficiency of large-scale SVM,this paper presented a new samples reduction method,called double-layer reduction method.It reduced data in two levels.The first level was coarse-grained reduction.It deleted the redundant clusters with KDC reduction.The second level was fine-grained reduction.It picked out the support vectors from the clusters remained by SMO.The experiments show that double-layer reduction method gives a higher compression ratio and accuracy.It applied the new method to the large scale SVM social spam detection model.The detection model speeds up obviously.Unlike the traditional reduction method,double-layer reduction method uses the concept of ＂Granularity＂ and ＂Level＂ into reducing method.It changes the reduced intensity according to the number of redundant points remained which has more advantage in reducing.