基于证据理论的单词语义相似度度量

Word Semantic Similarity Measurement Based on Evidence Theory

单词语义相似度度量一直是自然语言处理领域的经典和热点问题,其成果可对词义消歧、机器翻译、本体映射、计算语言学等应用具有重要影响.本文通过结合证据理论和知识库,提出一个新颖的度量单词语义相似度度量途径.首先,借助通用本体Word Net获取证据;其次,利用散点图分析证据的合理性;然后,使用统计和分段线性插值生成基本信任分配函数;最后,结合证据冲突处理、重要度分配和D-S合成规则实现信息融合获得全局基本信任分配函数,并在此基础上量化单词语义相似度.在数据集R&G(65)上,对比本文算法评判结果与人类评判结果的相关度,采用5折交叉验证对算法进行分析,相关度达到0.912,比当前最优方法 P&S高出0.4个百分点,比经典算法re LHS、dist JC、sim LC、sim L和sim R高出7%～13%;在数据集M&C(30)和Word Sim353上也取得了比较好的实验结果,相关度分别为0.915和0.941;且算法的运行效率和经典算法相当.实验结果显示使用证据理论解决单词语义相似度问题是合理有效的.

Measuring semantic similarity between words is a classical and hot problem in nature language processing, the achievement of which has great impact on many applications such as word sense disambiguation, machine translation, ontology mapping, computational linguistics, etc. This paper proposes a novel approach to measure words semantic similarity by combining evidence theory with knowledge base. Firstly, we extract evidences based on WordNet;secondly, we analyze the reasonableness of the extracted evidence using scatter plot;thirdly, we generate basic probability assignment by statistics and piecewise linear interpolation technique; fourthly, we obtain global basic probability assignment by integrating evidence conflict resolution, importance distribution, and D-S combination rules; finally, we quantify word semantic similarity. On data set R＆amp;G（65）, we conducted experiment through 5-fold cross validation, and the correlation of our experimental results with human judgment was 0.912, with 0.4% improvements over existing best practice P＆amp;S, 7%～13% improvements over classical methods （reLHS、distJC、simLC、simL, simR）; the experimental results based on M＆amp;C（30） and WordSim353 were also good with correlations being 0.915 and 0.941. The operational e？ciency of our method is as good as classical methods0, showing that using evidence theory to measure word semantic similarity is reasonable and effective.