两阶段域适应神经机器翻译方法

A two-phase domain adaptation method for neural machine translation

[目的]为了提升神经机器翻译模型的迁移学习效果,以语言数据为中心开展域适应方法探索.[方法]根据KL散度和最大均差两种域适应量度的定量分析结果,提出一种针对拥有大规模平行句子和小规模域文本场景的两阶段减量学习框架.第1阶段域过滤,利用域文本过滤平行句子,得到域平行句子,再利用得到的域平行句子训练出域神经机器翻译模型.第2阶段质量过滤,利用训练出的域神经机器翻译模型将第1阶段过滤出的域平行句子翻译一遍,比较机器译文与人工译文的质量,删除低质量平行句子以获得高质量域平行句子.最后利用得到的高质量域平行句子训练出优化的域神经机器翻译模型.[结果]在适应法律域英汉神经机器翻译上的实验结果显示新提出的两阶段算法只需原来训练步的四分之一左右,就反而可以提高2个多的BLEU分数.[结论]实验结论证明减量学习框架能够在大大减少训练时空开销的前提下获得最优的性能,最终实现神经机器翻译模型的快速域迁移.

[Objective]For the purpose of improving the transfer learning performance of neural machine translation(NMT)models,a domain adaptation method is explored with language data as the center.[Methods]According to the quantitative analysis results of two domain adaptation metrics of Kullback-leibler divergence and maximum mean discrepancy,a two-phase decremental learning framework is proposed for scenes with large-scale parallel sentences and small-scale domain texts.In the first phase,namely the domain filtering,domain texts are used to filter parallel sentences so that domain parallel sentences are obtained.Then,these obtained domain parallel sentences are used to train a domain NMT model.In the second phase,namely quality filtering,domain parallel sentences filtered in the first phase are translated by using the trained domain NMT model.Next,qualities of machine translation and manual translation are compared.Then,low quality parallel sentences are deleted to obtain high quality domain parallel sentences.Finally,an optimized domain NMT model is trained from the obtained high quality domain parallel sentences.[Results]Experimental results on English-Chinese NMT adapted to the legal domain show that the proposed two-phase algorithm only requires approximately a quarter of the original training steps,but can increase more than two BLEU points.[Conclusion]The experimental conclusion demonstrates that the decremental learning framework is capable of achieving the state-of-the-art performance with greatly reduced training space-time costs,and can implement fast domain transfer of NMT models.