基于非参控制图的文本分类方法

Text classification method based on non-parameter control graph

传统的文本分类方法多是基于词语本身.由于中文词语的复杂庞大,并且不断增加,直接用词语作为研究对象,容易造成文本特征向量的高度稀疏性和维数灾难,使得处理起来效率不高,难以计算.但是其词性的种类是固定不变的.随着中文分词和词性标注的研究越来越完善,使用词性作为文本分析的研究对象,吸引了越来越多公众和研究人员的兴趣.文章提出基于非参控制图的文本分类方法,仅使用句子中的各词性出现次数作为文本分类研究的特征值.利用中文自然语言处理平台,对句子进行词语切分以及词性标注,并保留出现次数最多的5个词性,计算每个词性在句子中的占比,进行等距对数比变换.最后把处理好的数据放入基于多元符号指数加权移动平均(MSEWMA)控制图模型中判别类型.该方法思想原理简单、容易实现、训练后处理速度快,通过实验证实可以很好地区分文本类别.

Traditional text classification methods are mostly based on words themselves.Due to the large and increasing complexity of Chinese words.Direct use of Chinese words as the research object is likely to cause highly sparse and dimensional disasters of text feature vectors.So the processing inefficiency is difficult to calculate.But,the number of parts of speech does not change.With the progress of Chinese word segmentation and part of speech tagging,use of part of speech as an object of study in text analysis has attracted growing interest from the general public and researchers.The purpose of this paper is to propose a text classification method based on non-parameter control graph,which only the frequency of occurrence of each part of speech in sentences is used as the eigenvalue to study text classification.By using the Chinese natural language processing platform,the sentences are segmented into words and marked with part of speech,and the 5 parts of speech that appear most often are retained.Calculate the proportion of each part of speech in a sentence and transforms the data by isometric log-ratio transformation.Finally,the processed data are put into a multivariate synthetic exponentially weighted moving average(MSEWMA)control chart to discriminate the types.The principle of this method is simple,easy to implement,and the processing speed after training is fast.Experimental results show that this method is able to distinguish text categories.