基于树状语料库的中文短语相似度计算

CHINESE SHORT SENTENCE SIMILARITY CALCULATION BASED ON TREE-STRUCTURE CORPUS

在诸如文件摘要、个性化搜索、学术诚信检测、常见问题自动解答、自动翻译等领域,短语相似度计算是核心算法。通过引入树状语料库,精确定义、计算词汇相似度,对基于抽取关键词序列的中文短语相似度算法做进一步改进。实验证明,该方法在提高中文短语相似度计算的准确率上达到了预期效果,更符合人的直观感觉。

In many fields, such as document summarisation, personalised searching, detection of academic integrity, FQA and automatic translation, the short sentence similarity calculation is the core algorithm. Through introducing the tree-structure corpus, we accurately define the similarity of words and calculate it, and make further improvement on the Chinese short sentence similarity algorithm based on keywords sequence extraction. Results of experiment show that this method achieves expected effect in improving the accuracy of Chinese short sentence similarity calculation, and is more in line with people＇s intuitive sense.