模板驱动的神经机器翻译

Template-Driven Neural Machine Translation

由于神经机器翻译模型简单、通用和有效,神经机器翻译模型已成为目前最受关注的机器翻译模型.在神经机器翻译模型中,通过引入词汇翻译表和短语翻译表可以提高翻译质量.然而,对于已经存在的人工整理的翻译模板或者启发式算法生成的翻译模板,目前已有的神经机器翻译框架不存在有效的方法对这些翻译模板进行建模.该文研究的主要内容是将翻译模板内嵌到端到端的神经机器翻译模型中.为此,我们提出了模板驱动的神经机器翻译模型,该模型通过使用额外的模板编码器对翻译模板进行端到端建模,通过使用知识门阀和注意力门阀动态地控制解码过程中不同来源的知识对当前解码词汇的贡献度的大小.知识门阀的主要作用是对源语言句子和翻译模板的信息进行有效的表示,从而更好地对解码器进行初始化.注意力门阀是一个基于时序的门阀,可以动态地控制当前翻译词汇接收源语言句子或者翻译模板信息的多少.最终实验结果表明,该文提出的方法对模板进行了有效的建模,20%词汇标准模板在汉英和英汉翻译任务上的翻译正确率分别高达93.6%和95.1%.与基线翻译系统相比,在汉英和英汉翻译任务上使用含有20%词汇的标准模板时,翻译性能可以增长4.2～7.2个BLEU值.当翻译模板中的真实词汇增加时,翻译质量得到进一步提升.

Nowadays,neural machine translation(NMT)has been the most prominent approach to machine translation(MT),due to its simplicity,generality and effectiveness.The principle of neural machine translation is to directly maximize the conditional probabilities of target sentences given source sentences in an end-to-end fashion.One of the most widely used neural machine translation model follows the encoder-decoder framework.It encodes the source sentence using a recurrent neural network(RNN)into a dense context representation,and produces the target translation from the context vector on the decoder.By exploiting the gating and attention mechanisms,neural machine translation models have been shown to surpass the performance of previously dominant statistical machine translation(SMT)on many well-established translation tasks.Recently,researchers have shown an increasing interest in incorporating external lexical translation table and phrase translation table into the neural machine translation,and obtained impressive translation performance.However,in the literature there is less study on incorporating translation templates,which are manually constructed or automatically induced by heuristic algorithm from parallel corpus,into the neural translation model.In this paper,we propose a novel architecture,template driven neural machine translation model,which extends to incorporate the additional translation template into the neural machine translation model.In contrast to the conventional neural machine translation model,on the source side,we use an additional recurrent neural network encoder(template encoder)to encode the additional translation template in parallel to the encoder for the source sentence.In our proposed template driven NMT model,firstly,we propose a gating mechanism,knowledge gate,to balance the information between the source sentence and the additional translation template that is best suited for inducing the source sentence representation.Secondly,to effectively leverage the knowledge representation in predicting the target words,we propose a weighted variant attention mechanism,attention gate,in which a time-dependent gating scalar is adopted to control the ratio of conditional information between the source sentence and the additional translation template.To evaluate the effectiveness of our proposal,we experiment with three kinds of translation templates:1)head template,where we preserve n words from the leftmost of a sentence and blank out the rest as slot to be predicted and filled by the neural machine translation model;2)tail template,where the leftmost words are blanked out by keeping the rightmost m words;and 3)normal template,the words are arbitrarily discarded to make slots to be filled by the translation model.Experimental results demonstrate that our proposed model can effectively make use of the additional information from the translation template,and the translation accuracy for the normal translation template with 20%of target words(of the sentence)is up to 93.6%and 95.1%on the Chinese-to-English and English-to-Chinese translation tasks,respectively.When we use 20%of target words as a translation template,we observe significant improvements of 4.2to 7.2BLEU scores compared with the baseline systems on the Chinese-to-English and English-to-Chinese translation tasks,respectively.Experiments also show that the translation performance goes up as more context words are considered in the translation template.