基于融合策略的机器翻译自动评价方法

A Novel MT Metric Based on the Hybrid Strategy

机器翻译自动评价发展至今,各种自动评价方法不断涌现。不同的自动评价方法从不同的角度评价机器译文的质量。该文提出了基于融合策略的自动评价方法,该方法可以融合多个自动评价方法,多角度地综合评价机器译文质量。该文主要在以下几个方面探索进行:(1)对比分别使用相对排序(RR)和直接评估(DA)两种人工评价方法指导训练融合自动评价方法,实验表明使用可靠性高的DA形成的融合自动评价方法(Blend)性能更好;(2)对比Blend分别使用支持向量机(SVM)和全连接神经网络(FFNN)机器学习算法,实验表明在当前数据集上,使用SVM效果更好;(3)进而在SVM基础上,探索使用不同的评价方法对Blend的影响,为Blend寻找在性能和效率上的平衡;(4)把Blend推广应用到其他语言对上,说明它的稳定性及通用性。在WMT16评测数据上的实验,以及参加WMT17评测的结果均表明,Blend与人工评价的一致性达到领先水平。

With the development of machine translation（MT）evaluation,various MT metrics have been proposed.Different metrics evaluate the quality of MT hypotheses from different perspectives.This paper proposes a novel MT metric that combines the merits of a range of metrics.Our investigation includes several aspects：（1）Comparing the performance of combined metrics that using Direct Assessment manual evaluation（DA）or Relative Ranking human evaluation（RR）to guide the training process.Experiments show that reliable DA human evaluation benefits the combined metric,Blend.（2）Comparing the performance of Blend using SVM or FFNN as the training algorithm.（3）Examining the contribution of metrics incorporated in Blend tentatively,in order to find a trade-off between performance and efficiency.（4）Applying Blend to other language pairs if incorporated metrics support the specific language pair.Experiments on WMT16 and WMT17 Metrics tasks show that Blend achieves the start-ofthe-art performance.