改进的基于知网的词语相似度算法

Improved word similarity algorithm based on HowNet

词语相似度计算在文本分类、问答系统、机器翻译、文本聚类等有着广泛的应用。词语相似度计算的研究工作一般都是基于《知网》的义原的层面上,根据义原之间的距离和义原本身的层次深度,进行词语相似度的计算。基于以上研究,提出了一种新的改进的词语相似度算法,首先根据义项中各类义原的个数不同,提出了一种新的变系数义项相似度计算方法;其次从词性的角度,认为词语义项中的不同词性对词语相似度的贡献度不同,剔除不同词性义项之间的组合。实验结果证明,改进的算法结果在原有基础上得到较好的提升,大幅度降低了相似度计算的复杂度,提高了运算效率。

The word similarity computation is widely used in text classification,question-answer system,machine-translation and text clustering.Research of this computation is generally based on HowNet,according to the distance and the depth of sememes.Based on above,an improved method of word similarity computation was proposed as follows.Firstly,a new variable coefficient of homonym similarity computing was proposed according to the count of homonym.Secondly,it took part of speech into account and argued that the part of speech of homonym is different in contributions to word similarity and removed the combinations of homonyms with different part of speech.The experimental results show that the result obtained through this newly-improved computation method is better with less complex calculation and higher calculation efficiency.