基于多阶段训练的跨语言摘要技术

Cross-Lingual Summarization Technology Based on Multi-stage Training

为解决跨语言摘要(Cross-Lingual Summarization,CLS)模型语义理解、跨语言对齐和文本生成能力不高的问题,提出了一个基于多阶段训练的英-中跨语言摘要模型。首先,进行多语言去噪预训练,同时学习中、英文的通用语言知识;其次,进行多语言机器翻译微调,同时学习对英文的语义理解、从英文到中文的跨语言对齐以及中文的文本生成能力;最后,进行CLS微调,进一步学习特定于CLS任务的语义理解、跨语言对齐和文本生成能力,最终获得一个性能优异的英-中跨语言摘要模型。实验结果表明所提模型的CLS性能有明显提升,且多语言去噪预训练和多语言机器翻译均可提高模型性能。与众多基线模型中的最优性能相比,所提模型在英-中跨语言摘要基准集上将ROUGE-1、ROUGE-2和ROUGE-L值分别提升了45.70%、60.53%和43.57%。

To solve the problem that the models of cross-lingual summarization(CLS)are poor in the semantic understanding,cross-lingual alignment and text generation,this paper proposes a CLS model based on the multi-stage training.Firstly,the model is trained by the multilingual denoising pre-training task,while learning common language knowledge in Chinese and English.Then,the model is trained by the multilingual machine translation task,simultaneously learning the following three types of abilities,semantic understanding of English,cross-lingual alignment from English to Chinese,and text generation of Chinese.Finally,the model is trained by the CLS task,further learning the above three types of abilities,eventually becoming an excellent English-to-Chinese CLS model.The experimental results show that the CLS performance of the proposed model is significantly improved,and the tasks of multilingual denoising pre-training and multilingual machine translation can both improve CLS performance.Experiments on an English-to-Chinese CLS benchmark dataset show that compared to the optimal performance in many baseline models,this model increases ROUGE-1,ROUGE-2 and ROUGE-L by 45.70%,60.53%and 43.57%,respectively.