基于混合采样和特征选择的改进随机森林算法研究

An improved random forest algorithm based on hybrid sampling and feature selection

随机森林算法是根据Bagging抽样和随机特征子集划分策略,由多棵决策树组成的集成算法。与其他分类算法相比,随机森林算法有更高的分类精度、更低的泛化误差以及训练速度快等特点,因此在数据挖掘领域得到了多方面的应用。然而随机森林算法在分类预测特征维度高且不平衡的数据时,分类性能受到了极大限制。为了更好地处理高维不平衡数据,文中提出了一种基于混合采样和特征选择的改进随机森林算法(Hybrid Samping&Feature Selection Random Forest,HF_RF)。该算法首先从数据层面出发,通过SMOTE算法和随机欠采样相结合的方式对高维不平衡数据集进行预处理,同时引入聚类算法对SMOTE算法进行改进,提高对负类样本的处理性能;然后从算法层面出发,通过ReliefF算法对平衡后的高维数据赋予不同的权值,剔除不相关和冗余特征,对高维数据进行维度约简;最后采用加权投票原则进一步提高算法的分类性能。实验结果显示,改进后的算法与原算法相比,在处理高维不平衡数据方面的各评价指标更高,证明HF_RF算法对于高维不平衡数据的分类性能高于传统随机森林算法。

The random forest algorithm is an integration of multiple decision trees based on the Bagging sampling and the random feature subset division strategy.Compared with other classification algorithms,the random forest algorithm can provide higher classification accuracy,lower generalization error and faster training speed,so it has been applied to many fields involving data mining.However,it performs unsatisfactorily on imbalanced data with high dimensional features.Therefore,this paper proposes an improved random forest algorithm named HF_RF based on hybrid sampling and feature selection.First,the algorithm preprocesses the unbalanced data with high dimensional features from the data level by combining the SMOTE algorithm and the random under-sampling.In the SMOTE algorithm,the clustering algorithm is introduced to improve the capability of dealing with negative samples.Second,from the algorithm level,the ReliefF algorithm is used to assign different weights on the high-dimensional data that have been balanced,and eliminate the irrelevant and redundant features.Thus,the dimensions of the data are reduced.Third,the weighted voting principle is used to further improve the classification ability.The evaluation experiments show that,the proposed algorithm achieves better results in processing high-dimensional unbalanced data,compared with the original random forest algorithm.This demonstrates that the proposed HF_RF algorithm performs better on high-dimensional unbalanced data than the traditional random forest algorithm.