中文短文本自动分类中的汉字特征优化研究

Research on the Optimization of Chinese Character Features in the Automatic Classification of Chinese Short-text

采用含语义的词语或篇幅更长的语言片段作为中文短文本的特征描述存在明显的可操作性问题。文章综合探讨了汉字特征在中文短文本分类计算中的可行性以及影响规律,比较了关键词、词语和汉字的类目区分能力,认为后者的分类效果略低于篇幅大的语言片段,但其具有可计算性强和文本覆盖率高的优点;基于类现频次和信息增益复合方法对汉字特征进行了筛选,总结了汉字特征数量减少对分类效果的影响规律;分析了不同特征权重设置对汉字特征分类效果的影响及其原因,认为汉字在词语中的位置参数及其频次参数的有效结合可以在一定程度上提高汉字特征的分类效果。

It has significant operability problems that using words with clear meanings or language fragments much longer as descriptive features for Chinese short-text. This paper comprehensively discussesfeasibility and influence rules of character features applied to classification calculation for Chinese short-texts, compares the category distinguishing ability of keywords, terms and Chinese character. The paper indicates that character features, which performance slightly worse than longer language fragment, have the advantage of stronger calculability and higher text coverage. The paper screens character features by a method with composi- tion of occurrence frequency in categories and information gain, and summarizes the influence rules of the decrease in the number of Chinese character features for classification effect. Based on the analysis of the impact on classification effectiveness of character fea- tures with different setting schemes for features weight and its reasons, the paper believes that the approach of Chinese character po- sition in words effectively integrated frequencies factors could improve classification performance of Chinese character features to some extent.