基于约束优化的神经机器翻译系统研究

Research on neural machine translation system based on constrained optimization

针对传统神经机器翻译模型结构复杂,参数规模巨大导致的参数学习困难的问题,研究提出一种基于检查点约束的神经机器翻译系统。通过采用知识蒸馏的方法,构造了神经机器翻译模型的约束函数;然后根据模型不断产生的检查点构建了新的教师模型,实现了动态调整约束。最后,在NIST汉语-英语和WMT17汉语-英语数据集上对本方法进行验证。结果表明,相较于传统基线模型和基于网格搜索的参数优化模型,本系统采用的基于检查点约束的神经机器翻译方法,可有效提高模型训练效果,缓解模型训练过程中过拟合问题,具有更优良的性能。

traditional neural machine translation(NMT)model has complex structure and large scale of parameters,which makes it difficult to learn parameters.By using the method of knowledge distillation,this paper constructs the constraint function of neural machine translation model,and then constructs a new teacher model according to the checkpoints generated by the model,which realizes the dynamic adjustment of constraints.Finally,the system is validated on NIST Chinese English and wmt17 Chinese English datasets.The results show that compared with the traditional baseline model and the parameter optimization model based on grid search,the neural machine translation method based on checkpoint constraint can effectively improve the model training effect,alleviate the over fitting problem in the model training process,and has better performance.