动态向量的中文短文本聚类

Chinese short text clustering based on dynamic vector

因中文短文本特征词词频低、存在大量变形词和新词的特点,使得中文短文本相似度发生漂移,难以直接使用现有的面向长文本的聚类算法。针对短文本相似度漂移问题,提出了一种基于《知网》扩充相关词集构建动态文本向量的方法,利用动态向量计算中文短文本的内容相似度,进而发现短文本之间的内在关联,从而缓解特征词词频过低和存在变形词以及新词对聚类的影响,获得较好的聚类结果。实验结果表明,该算法的聚类质量高于传统算法。

Since Chinese short text is short of keywords and full of anomalous words,it brings about short text similarity drift and the traditional text clustering method is not directly suitable for short text clustering.To solve the problem of sparse key-words and similarity drift in short text segments,this paper proposes a new method to build dynamic text vector by text similarity based on HowNet.This method can measure the similarity between short text segments by dynamic text vector,then find short text relationship,so as to relieve these two characteristics＇bad influence on the clustering performance and therefore to gain a better clustering result.Experiments show the method can get better performance in Chinese short text clustering,compared with traditional method.