低频词表示增强的低资源神经机器翻译

Low Resource Neural Machine Translation with Enhanced Representation of Rare Words

在神经机器翻译过程中,低频词是影响翻译模型性能的一个关键因素。由于低频词在数据集中出现次数较少,训练经常难以获得准确的低频词表示,该问题在低资源翻译中的影响更为突出。该文提出了一种低频词表示增强的低资源神经机器翻译方法。该方法的核心思想是利用单语数据上下文信息来学习低频词的概率分布,并根据该分布重新计算低频词的词嵌入,然后在所得词嵌入的基础上重新训练Transformer模型,从而有效缓解低频词表示不准确问题。该文分别在汉越和汉蒙两个语言对四个方向上分别进行实验,实验结果表明,该文提出的方法相对于基线模型均有显著的性能提升。

In neural machine translation,the low-frequency word is a key factor affecting the quality of the translation output,which is more prominent in low-resource scenario.This paper proposes a low-resource neural machine translation method with enhanced the representation of low-frequency words.The main idea is to use monolingual data context information to learn the probability distribution of low-frequency words,and recalculate the word embeddings of low-frequency words based on this distribution.The Transformer model is then re-trained by the new word embeddings,thereby effectively alleviating the problem of representing low-frequency words inaccurately.The experimental results in the four directions between Chinese and Vietnamese,Chinese and Mongolian translation tasks show that the method proposed in this paper has a significant improvement over the baseline model.