基于遗传规划集成学习的网络作弊检测

Web Spam Detection by the Genetic Programming-based Ensemble Learning

网络作弊检测是搜索引擎的重要挑战之一,该文提出基于遗传规划的集成学习方法 (简记为GPENL)来检测网络作弊。该方法首先通过欠抽样技术从原训练集中抽样得到t个不同的训练集;然后使用c个不同的分类算法对t个训练集进行训练得到t*c个基分类器;最后利用遗传规划得到t*c个基分类器的集成方式。新方法不仅将欠抽样技术和集成学习融合起来提高非平衡数据集的分类性能,还能方便地集成不同类型的基分类器。在WEBSPAM-UK2006数据集上所做的实验表明无论是同态集成还是异态集成,GPENL均能提高分类的性能,且异态集成比同态集成更加有效;GPENL比AdaBoost、Bagging、RandomForest、多数投票集成、EDKC算法和基于Prediction Spamicity的方法取得更高的F-度量值。

Web spam detection is a challenging issue for web search engines. This paper proposes a Genetic Program- ming-based ensemble learning approach （GPENL） to detect web spare. First, the method gets t different training sets by the under-sampling from the original training set. Then, c different classification algorithms are used on t training sets to get t ＊ c base classifiers. Finally, an integrated approach of t ＊ c base classifiers is obtained by Genet- ic Programming. The new method can not only merge the under-sampling technology and ensemble learning to im- prove the classification performance on imbalanced datasets, but also conveniently integrate different types of base classifiers. The experiments on WEBSPAM-UK2006 show that this method improve the classification performance whether the base classifiers belong to the same type or not, and in most cases the heterogeneous classifier ensembles work better than the homogeneous ones and GPENL can get higher F-measure than those clone by AdaBoost, Bag- ging, RandomForest, Vote, EDKC algorithm and the method based on Prediction Spamicity.