融合多层注意力表示的中文新闻文本摘要生成

CHINESE NEWS TEXT SUMMARIZATION GENERATION BASED ON MULTI-LEVEL ATTENTION REPRESENTATION

针对现有的文本摘要模型词向量表意不全面,且难以对远距离词句进行表征,提出一种融合多层注意力表示的中长文本摘要方法。通过抽取式技术将新闻文本进行分割,得到主体文本和辅助文本;将主体文本进行图卷积神经网络的学习和依存句法分析,得到词向量的图卷积表示和依存词对信息,同时对辅助文本进行高频主题词的挖掘;将这三种信息融合送入Transformer序列模型中,并对编码器和解码器的局部注意力稍作修改,使其能够更多地关注主题相关的部分和依存句法结构;生成文本摘要。在公共文本摘要数据集NLPCC 2017上的实验表明,该方法能够得到较高的ROUGE分数,生成质量更好的文本摘要。

In view of the fact that the word vector of the existing text summarization model is not comprehensive and difficult to represent the long-distance words and sentences,this paper proposes a medium long text summarization method with multi-level attention representation.The news text was segmented by extractive technology to get the main text and auxiliary text.The main text was studied by graph convolution neural network and dependency syntax analysis to obtain the graph convolution representation of word vector and the information of dependent word pairs.At the same time,the auxiliary text was mined with high-frequency topic words.These three kinds of information were fused into the transformer sequence model.The local attention of the encoder and decoder was modified to make them pay more attention to the topic related parts and dependency syntactic structure.The text summary was generated.Experiments on the public text summarization dataset NLPCC 2017 show that this method can get higher ROUGE scores and generate better quality text summaries.