基于Transformer和重要词识别的句子融合方法

THE METHOD OF SENTENCE FUSION BASED ON TRANSFORMER AND KEYWORD RECOGNITION

句子融合是为多个句子生成言简意赅、符合语法的句子,可应用到自动摘要、复述生成等自然语言处理任务。目前句子融合方法已取得一定成效,但还存在重要信息缺失、语义偏离原句等问题。该文提出基于Transformer和重要词识别的方法来缓解上述问题。该方法包括两个模块,(1)重要词识别模块:利用序列标注模型识别原句重要词;(2)句子融合模块:将重要词与原句输入到Transformer框架并利用BERT进行语义表示,然后在全连接层引入基于原句和词表获得的向量作为先验知识进行句子融合。基于NLPCC2017摘要任务集构建句子融合数据集,并进行相关实验,结果表明所提方法的性能明显优于基线系统。

Sentence fusion is to generate the concise,grammatical sentence for multiple sentences,which can be applied to many natural language processing tasks such as automatic summary and paraphrase generation.At present,the sentence fusion methods have achieved certain results,but the generated sentence still has the problems such as missing some important information or deviating from the semantics of the original sentences.In order to alleviate above problems,the paper proposes a sentence fusion method based on Transformer and keywords recognition.The method includes two modules.(1)Keyword recognition module used the sequence labeling model to recognize keywords of the original sentences.(2)Sentence fusion module took the keywords and the original sentences into the Transformer framework,and got the semantic representation with BERT.Then it introduced the vector,which was obtained based on the original sentences and the vocabulary in the fully connected layer,as the prior knowledge to generate fusion sentences.Based on the NLPCC2017 summary task set,we constructed sentence fusion set,and conducted related experiments.The results show that the performance of the proposed method is significantly better than the baseline system.