ROUGE-SN:基于跨越N元语法的机器翻译评测方法

ROUGE-SN: A Machine Translation Evaluation Method Based on Skip-Ngram

本文在ROUGE-S的基础上提出一种基于跨越N元语法的ROUGE-SN机器翻译评测方法,在跨越二元语法(Skip-bigram)的基础上尽量延长N元语法的长度,使更多的句子连贯信息得以体现。并通过设置N元语法的阈值、综合系统运行代价和译文匹配效率等因素选定ROUGE-S6对ROUGE-S进行改进。在俄汉双语句子数据集上对谷歌、百度、必应、有道在线翻译系统的俄汉翻译输出译文进行评测,ROUGE-S6方法与传统ROUGE-S以及BLEUS的评测结果一致且性能优于ROUGE-S和BLEUS;且基于跨越N元语法的ROUGE-S6使得ROUGE-S的性能得以提升,对于百度系统而言,ROUGE-S性能提升44.52%,对于谷歌系统而言,提升50.45%,对必应系统提升42.19%,有道系统中ROUGE-S性能提升40.01%。

This paper addressed a new machine translation evaluation method ROUGE-SN based on Skip-Ngram. This new method tried to extend the length of N-gram on the basis of Skip-bigram, to embody more sentence coherence information. What's more, it set Nth as the threshold value of N-gram, and selected ROUGE-S6 to improve the ROUGE-S performance considering the running cost of the system and the translations matching efficiency. Experiments were performed in Russian and Chinese bilingual sentence data set and it evaluated the output translations of online translation system such as Google, Baidu, Bing and Youdao. The evaluation results of ROUGE-SN were consistent with that of ROUGE-S and BLEUS, and the performance of ROUGE-SN was the best in the three. The experiment results also showed that the ROUGE-S6 based on Skip-Ngram improve the ROUGE-S performance. For Baidu, the ROUGE-S performance improved 44.52%; Google was 50.45%; Bing 42.19% and Youdao 40.01%.