基于BERT_Att的机器阅读理解模型

MACHINE READING COMPREHENSION MODEL BASED ON BERT_ATT

针对当前机器阅读理解模型中文本与问题的语义融合不够充分、缺乏考虑全局的语义信息的问题,提出一种基于BERT、注意力机制与自注意力机制的机器阅读理解模型BERT_Att。该模型采用BERT将文本和问题分别映射进特征空间,通过Bi-LSTM、注意力机制与自注意力机制将文本与问题进行充分的语义融合,通过Softmax计算答案的概率分布。在公共数据集DuReader上的实验结果表明,该模型的BLEU-4值与ROUGE-L值较现有的模型均有进一步的提升,并且分析了影响模型表现的因素,验证了该模型设计的有效性。

Aimed at the problem that the semantic fusion of the passages and the problems in the current machine reading comprehension model was not sufficient and the global semantic information was not considered,a machine reading comprehension model BERT_Att based on BERT,attention mechanism and self-attention mechanism is proposed.This model used BERT to map passages and problems into feature space.Through Bi-LSTM,attention mechanism and self-attention mechanism,the semantic fusion of passages and problems was fully carried out.Softmax was used to calculate the probability distribution of the answers.Experimental results on the public dataset DuReader show that BLEU-4 value and ROUGE-L value of this model are further improved compared with the current models.By analyzing the factors that affect the performance of this model,the effectiveness of this model is verified.