基于二阶段对比学习的中文自动文本摘要方法研究

Automatic Chinese Text Summarization Based on Two-Stage Contrastive Learning

在中文自动文本摘要中,暴露偏差是一个常见的现象。由于中文文本自动摘要在序列到序列模型训练时解码器每一个词输入都来自真实样本,但是在测试时当前输入用的却是上一个词的输出,导致预测词在训练和测试时是从不同的分布中推断出来的,而这种不一致将导致训练模型和测试模型直接的差异。本文提出了一个两阶段对比学习框架以实现面向中文文本的生成式摘要训练,同时从摘要模型的训练以及摘要评价的建模进行对比学习。在大规模中文短文本摘要数据集(LCSTS)以及自然语言处理与中文计算会议的文本数据集(NLPCC)上的实验结果表明,相比于基线模型,本文方法可以获得更高的面向召回率的摘要评价方法(ROUGE)指标,并能更好地解决暴露偏差问题。

Exposure bias is a common phenomenon in Chinese automatic text summarization.Seq2Seq models are usually trained by using the teacher forcing method in a maximum likelihood framework.During the decoding phase of training and validation,the true summaries are fed into the decoder.Each word input to the decoder of Chinese text auto-summarization comes from real samples during training.The current input is used as the output of the previous words during testing.It leads to the prediction words being inferred from different distributions during training and testing,leading to the direct difference between the training model and the testing model.A two-stage comparative learning framework is proposed to implement generative summary training for Chinese text.Contrastive learning is also performed from the training of summary models as well as the modeling of summary evaluation.A series of rigorous experimental designs and statistical analysis methods are employed to comprehensively evaluate and validate the text summarization method proposed in this paper.The experimental results show that the proposed method achieves significant improvements in both summary quality and efficiency compared with traditional text summarization methods.Contrastive learning from the training of summary models as well as from the modeling of summary evaluation are simultaneously conducted.In addition,rather than constructing a positive or negative example as in most existing comparative learning work,the"comparability"of this paper is reflected in the different qualities of the naturally generated summarization.The experimental results show that the proposed method has better performance on the LCSTS and NLPCC datasets.