一种基于语义相似度的文本聚类算法

Clustering Method Based on Semantic Similarity

文本聚类在很多文本挖掘和信息检索系统中发挥着重要的作用。现有的聚类算法大多数都是基于向量空间模型,文档集合中出现的单词词频作为特征项。这些算法都存在数据维数过高、聚簇难以描述的问题,而且忽略了单词间的语义联系。本文提出了一种基于语义相似度的文本聚类算法——TCU SS(Text clustering usingsem an ticsim ilarity)算法。TCU SS算法将文档表示成概念列表,有效地解决了数据维数高和聚簇描述难的问题,并给出如何利用概念列表进行聚簇描述的方法。TCU SS算法利用两个概念列表中单词间的语义相似度作为文档间相近程度的度量,并以图为基础进行聚类分析,避免有些聚类算法对聚簇形状的限制。实验证明,TCU SS算法提高了聚类质量。

Common document clustering algorithms rely on the so-called vector space model using the item frequency as the feature. However, these methods donot really address the special problems of text clustering： high dimensionality of the data and understandability of the cluster description. Moreover, words may be semantically related ——a crucial information for clustering does not considered. A new document clustering method based on semantic similarity text clustering using semantic similarity （TCUSS） is proposed. TCUSS algorithm uses documents as concept lists to solve the problems mentioned before and gives a method how to describe the clusters by concept lists. TCUSS algorithm mea- sures the document similarity by semantic similarity of concepts in concept lists, then clusters the document based on graph analysis, thus avoiding the restrict of clusters shape. Experimental results prove that TCUSS algorithm improves the quality of the clusters.