文本分类算法中词语权重计算方法的改进

Modify the Method of Feature's Weight in Text Classfication

在自动文本分类中,TFIDF公式是常用的词语权重计算公式。该方法简单易行,但仅仅考虑了特征词出现的频率,而忽略了特征词对区分每个类的贡献。针对这个不足,该文提出了TFIDF-CHI,来修正各个特征词的权重,重新调整每个特征词对各个类别的区分度,并用KNN分类器来验证其有效性。实验证明该方法优于原来的TFIDF算法,表明了改进的策略是可行的。

In auto text classification,TFIDF is often used when the weight of a term is calculated.The method is easy,only considers the frequency of the feature and ignores the feature's contribution to each class.Aiming at this shortage,we put forward the TFIDF-CHI and use it to modify each feature's weight,read just each feature's differentiation to each class.Then the KNN classifier is used to check its validity.The method is better than traditional TFIDF and proves that the TFIDF-CHI method is feasible.