文本自动分类中特征权重算法的改进研究

An Improved Approach to Term Weighting in Automated Text Classification

文章研究并改进了文本自动分类中的特征权重算法。传统的特征权重算法着重于考虑频率和反文档频率等因素,而未考虑特征的类间、类内分布与低频高权信息。该文重点研究了特征的类间、类内分布,以及低频高权特征对分类的影响,并在此基础上提出了低频高权特征集的构造方法及特征权重的新算法,同时将该算法推广到多层次分类体系。实验证明该算法能有效提高分类的精确度,而且在多级分类中也能取得很好的效果。

This article aims to improve the algorithm of term weighting in automated text classification.Traditional algorithms only consider about TF(Term Frequency),IDF(Inverse Document Frequency)and so on,and do not consider DI(Distribution Information) among and inside classes and LFHW(Low Frequency but High Weight) terms.This article mainly researches about the impassion of DI and LFHW terms on classification,the construction of LFHW term sets and new approaches to term weighting.These new approaches are also applied to the hierarchical classification system.The comparison of experimental results proves that these new approaches can not only improve the precision of classification,but also have a good performance in hierarchical classification.