基于双向对比训练的中文故事结尾生成模型

Chinese story ending generation model based on bidirectional contrastive training

中文故事结尾生成(SEG)是自然语言处理中的下游任务之一。基于全错误结尾的CLSEG(Contrastive Learning of Story Ending Generation)在故事的一致性方面表现较好。然而,由于错误结尾中也包含与原结尾文本相同的内容,仅使用错误结尾的对比训练会导致生成文本中原结尾正确的主要部分被剥离。因此,在CLSEG基础上增加正向结尾增强训练,以保留对比训练中损失的正确部分;同时,通过正向结尾的引入,使生成的结尾具有更强的多样性和关联性。基于双向对比训练的中文故事结尾生成模型包含两个主要部分:1)多结尾采样,通过不同的模型方法获取正向增强的结尾和反向对比的错误结尾;2)对比训练,在训练过程中修改损失函数,使生成的结尾接近正向结尾,远离错误结尾。在公开的故事数据集OutGen上的实验结果表明,相较于GPT2. ft和深层逐层隐变量融合(Della)等模型,所提模型的BERTScore、METEOR等指标均取得了较优的结果,生成的结尾具有更强的多样性和关联性。

Chinese Story Ending Generation(SEG)is one of the downstream tasks in Natural Language Processing(NLP).CLSEG(Contrastive Learning of Story Ending Generation)based on completely wrong endings performs well in terms of story consistency.However,due to the fact that the wrong ending also contains the same content as the original ending text,using only the wrong ending for contrastive training may results in the main part of the generated text with the correct ending being stripped off.Therefore,forward ending enhancement training was added on the basis of CLSEG to preserve the correct parts lost in contrastive training.At the same time,by introducing forward endings,the generated endings have stronger diversity and relevance.The proposed Chinese story ending generation model based on bidirectional contrastive training consisted of two main parts:1)multi-ending sampling,by which positively enhanced endings and reverse contrasted erroneous endings were obtained by different model methods;2)contrastive training,by which the loss function was modified during the training process to make the generated ending close to the positive ending and away from the wrong ending.Experimental results on the publicly available story dataset OutGen show that compared to models such as GPT2.ft and Della(Deeply fused layer-wise latent variable),the proposed model achieves better results in BERTScore,METEOR,and other indicators,generating more diverse and relevant endings.