摘要
在文本自动分类中,目前有词频和文档频率统计这两种概率估算方法,采用的估算方法恰当与否会直接影响特征抽取的质量与分类的准确度。本文采用K最近邻算法实现中文文本分类器,在中文平衡与非平衡两种训练语料下进行了训练与分类实验,实验数据表明使用非平衡语料语料时,可以采用基于词频的概率估算方法,使用平衡语料语料时,采用基于文档频率的概率估算方法,能够有效地提取高质量的文本特征,从而提高分类的准确度。
In text categorization,there are two kinds of probability estimation methods that they based on word frequency and document sfrequency,the stimation method' approiproteion or not will directly affect on the quality of feature extraction and classification accuracy.In this paper,there are train and classification experiment in the balanced and unbalanced China training data based on K-nearest neighbor algorithm as Chinese text classifier,the experimental data show that there are good effect on text feature extraction when word frequency is used in balanced data and document frequency is used in inbalanced data,which it can get high-quality texts,and improve the accuracy of classification.
出处
《情报科学》
CSSCI
北大核心
2011年第4期544-547,共4页
Information Science
基金
国家社科基金项目(编号:08CTQ003)
基张家口市科科学技术研究与发展项目(编号:0921045B)
河北北方学院自然科学青年基金项目(编号:Q2010008)
关键词
分本分类
特征抽取
概率估算
classification
features extraction
probability estimate