一种改进的文本特征选择方法的研究与设计

STUDY AND DESIGN OF AN IMPROVED TEXT FEATURE SELECTION METHOD

分析了特征选择采用互信息方法时文本分类性能较低的原因,认为与其在特征选择时倾向于选择稀有特征这一缺陷有很大关系。在此基础上,提出了一种基于分散度和平均频度的互信息特征选择方法。实验结果表明,改进后的互信息方法使文本分类性能有明显提高。

The article explains why text classification performance is low when mutual information method is adopted in feature selection,asserts that it is largely due to the flaw of selection of rare feature when making feature selections.Next a mutual information feature selection method based on distributed degree and average frequency is proposed.Experimental results show that the improved mutual information method can significantly improve the text classification performance.