基于维基百科社区挖掘的词语语义相似度计算

Semantic Similarity Computing Based on Community Mining of Wikipedia

词语语义相似度计算在自然语言处理如词义消歧、语义信息检索、文本自动分类中有着广泛的应用。不同于传统的方法,提出的是一种基于维基百科社区挖掘的词语语义相似度计算方法。本方法不考虑单词页面文本内容,而是利用维基百科庞大的带有类别标签的单词页面网信息,将基于主题的社区发现算法HITS应用到该页面网,获取单词页面的社区。在获取社区的基础上,从3个方面来考虑两个单词间的语义相似度:(1)单词页面语义关系;(2)单词页面社区语义关系;(3)单词页面社区所属类别的语义关系。最后,在标准数据集WordSimilarity-353上的实验结果显示,该算法具有可行性且略优于目前的一些经典算法;在最好的情况下,其Spearman相关系数达到0.58。

Words semantic similarity computing has been widely used in natural language processing, such as word sense disambiguation, information retrieval, text auto categorization. Different from traditional methods, we presented an algo- rithm based on community mining of Wikipedia to compute words semantic similarity. Our method makes use of the huge Wikipedia page network with category labels rather than its textual content. To get the community of a word page,we applied the HITS,which is a community discovery algorithm based on the theme, to pages network. Based on the gotten community,we measured the semantic similarity between two words from three aspects： （1）semantic rela- tions between the two word pages, （2）semantic relations between the two communities of word page, （3）semantic rela- tions between the categories which two communities belong to. Finally, tests on standard data sets WordSimilarity-353 show that the method we proposed is feasible and slightly better than some classic algorithms. In the best case, the Spearman correlation coefficient reaches 0. 58.