一种新的句子相似度度量及其在文本自动摘要中的应用

A New Method for Calculating Similarity Between Sentences and Application on Automatic Text Summarization

本文提出了一种新的句子相似度度量的方法并应用于文本自动摘要中。其创新处在于相似度计算不仅考虑句子中的uni gram ,还考虑了bi gram和tri gram ,通过回归方法将这几种相似度结果综合起来。实验证明这种相似度计算方法是有效的。同时本文还提出了一种新的 ,利用句子间相似度以及句子的权重的抽句式文摘算法 ,在抽取出句子的同时也去掉了冗余。DUC2 0 0 3、DUC2 0 0 4 (DocumentUnderstandingConference 2 0 0 3,2 0 0 4 )的评测结果征明了方法的有效性。我们的系统在DUC2 0 0 4的评测中列第二位。

This paper introduces a new method for calculating similarity between sentences.The algorithm uses not only uni gram but also bi gram and tri gram to calculate similarity.The algorithm is based on regression methods. Experimentations show that the method effective.The final summarization result is better than the algorithm that does not use it.We also propose a new summarization algorithm based on sentences weight and the new sentence similarity calculating method.While extracting the most important sentences,redundancy is also reduced.The evaluation of DUC2003 and DUC2004 shows its effectiveness.Our system rank second among all systems that join in the DUC 2004.