基于向量空间模型的文本分类系统的研究与实现

Research and Implementation of Text Classification System Based on VSP

文本分类是信息处理的一个重要的研究课题 ,它可以有效的解决信息杂乱的现象并有助于定位所需的信息。本文综合考虑了频度、分散度和集中度等几项测试指标 ,提出了一种新的特征抽取算法 ,克服了传统的从单一或片面的测试指标进行特征抽取所造成的特征“过度拟合”问题 ,并基于此实现了二级分类模式的文本分类系统。和类中心分类法相比 ,实验结果表明二级分类模式具有较高的精度和召回率。

Text classification is an important research task of natural language processing, which can efficiently resolve the issue of information chaos and help to locate the required information. The traditional approaches of text classification commonly extract feature terms from a single test criterion, which will lead to the problem of “over fitting'. This paper comprehensively takes test criterions such as frequency, distribution and concentration into account and proposes a new arithmetic of feature extraction and implements text classification system with two-level mode. The experimental results show that two-level classification mode has higher classification precision and recall compared with center classification method.