基于词向量和多特征语义距离的文本聚类算法

Text Clustering Algorithm Based on Word Vector and Multi-feature Semantic Distance

针对传统文本聚类算法中存在语义相似度计算向量维度高、忽视特征词词频、位置、词距和语义缺失等问题,提出了一种基于词向量和多特征语义距离的文本聚类算法(M-W2-KS)。首先使用Word2Vec训练语料库中的所有特征词,以向量形式进行表征;然后综合考虑特征词的词频、位置、词距信息以及特征词间的欧式距离,计算文本间的语义相似度,并将其应用到K-means算法中,实现文本的聚类。实验结果表明,运用M-W2-KS算法,可以有效提升聚类效果,使聚类结果更加准确。

The traditional text clustering algorithm ignores high dimension of semantic similarity calculation vectors, word frequency, location, word distance and semantic missing of feature words. Therefore, a text clustering algorithm based on word vector and multi-feature semantic distance (M-W2-KS) is proposed. Firstly, all feature words in word2Vec training corpus are represented as vectors, and then the semantic similarity between texts is calculated by combining the frequency, location, distance and Euclidean distance of feature words. Finally, it is applied to K-means algorithm to realize text clustering. The experiment results show that M-W2-KS algorithm can effectively enhance clustering results, and make clustering results more accurate.