基于信息熵加权的Word2vec中文文本分类研究

Research on Chinese Text Classification Based on Word2vec

针对中文文本分类中文本向量表示和词汇重要性等问题,提出基于Word2vec的中文文本分类方法。首先采用Word2vec训练生成文本向量;然后根据信息熵的概念,计算出不同词汇在文档中的重要程度,对向量进行加权;最后使用SVM分类器对加权后的词向量进行训练。实验结果表明,本文提出的分类方法在精确率、召回率和F-measure均有显著提高,具有较好的分类效果。

Aiming at the problems of text vector representation and vocabulary importance in Chinese text classification,this paper proposes a Chinese text classification method based on Word2vec.Firstly,Word2vec is used to generate text vectors.Then,according to the concept of information entropy,the importance of different words in the document is calculated and the vector is weighted.Finally,the weighted word vector is trained by SVM classifier.The experimental results show that the classification method proposed in this paper has a significant improvement in accuracy,recall rate and F-measure,and a good classification effect.