基于义原知识和双向注意力流的问题生成模型

Question Generation Based on Sememe Knowledge and Bidirectional Attention Flow

【目的】为解决现有模型存在的生成问题语义偏离于给定上下文文本和答案的问题,提出一种基于义原知识和双向注意力流的问题生成模型。【方法】提出两种语义增强策略:(1)通过在嵌入层融入义原外部知识的方法来捕捉比词向量更小粒度的语义知识,进而增强文本自身的语义特征。此外,通过余弦相似度算法得到更符合上下文文本语义的扩充义原知识库,这样做不仅可以筛除原有义原知识库中可能会导致语义嘈杂的义原,而且可以为词表中无义原标注的单词推荐符合语义的义原集合。(2)通过在编码层后融入双向注意力流的方法,增强文本与答案之间的语义表征。【结果】本模型在SQuAD1.1数据集上的Bleu_1、Bleu_2、Bleu_3、Blue_4评价指标分别达到46.70%、31.07%、22.90%、17.48%。实验证明,本文所提改进模型性能优于基线模型。【局限】当融入双向注意力流时,由于模型需要分别对段落文本及问题进行特征提取,因此训练模型时需要消耗成倍的内存和时间。【结论】义原知识和双向注意力流这两种语义增强策略可以增强问题生成模型的效果,并且使模型生成更符合人类语言习惯的更高质量的问题。

[Objective] This paper proposes a question generation model based on sememe knowledge and bidirectional attention flow, aiming to improve the semantics of the questions. [Methods] We developed two strategies to enhance semantics:(I) By integrating the external knowledge of sememe in the embedding layer, we captured the semantic knowledge with a smaller granularity than word vectors, and then enhanced the semantic features of the text itself. In addition, we obtained an expanded sememe knowledge base that is more in line with the semantics of the contextual text through the cosine similarity algorithm. It helped us filter out the sememes creating semantic noise in the original knowledge base, and recommended semantically compliant sememe sets for words labeled with non-semantic origins.(II) We enhanced the semantic representation between texts and answers by incorporating a bidirectional attention flow after the encoding layer. [Results] We evaluated our model with the SQuAD1.1 dataset, and the Bleu_1, Bleu_2, Bleu_3, and Blue_4 reached 46.70%, 31.07%, 22.90%, and17.48%, respectively. The proposed model outperformed the baseline models. [Limitations] With the bidirectional attention flow, the model needs to extract features of paragraph texts and questions, which demands double memory and time to train the model. [Conclusions] Sememe knowledge and bidirectional attention flow could help the proposed model generate higher-quality questions more in line with human language habits.