藏文文本分类特征基元选择

Feature Primitives Selection for Tibetan Text Classification

文本分类是指根据预先定义的主题类别,按照一定的规则将文档集合中未知类别的文档按内容自动确定某种类别的技术,是自然语言处理中最基本、最重要的研究内容之一,在信息检索、智能推荐、舆情分析、新闻分类等领域具有极高的应用价值。现阶段,藏文文本分类的主要研究集中在借用英汉等语言文本分类技术构建分类器,英汉等语言以词做分类特征基元构建分类器。由于受藏文分词技术制约,直接以词做藏文文本分类特征基元,其性能有较大的影响。该文在分析文本分类流程及藏文文本构成的基础上,研究了藏文文本分类特征基元选择方法,提出了一种融合词和音节的藏文文本分类特征基元选择方法,并以CNN模型构建的分类器验证了该方法的有效性。

Text classification refers to automatically determining a document for its category according to the predefined topic categories.At present,the most existing researches on Tibetan text classification are focused on using English and Chinese text classification technology to build classifiers,which are usually based on word features.Due to the restriction of Tibetan word segmentation technology,the performance of Tibetan text classification is greatly affected by using word features.This paper proposes a feature primitives selection method for Tibetan text classification by combining words and syllables,and the classifier constructed by CNN model verifies the effectiveness of the method.