基于池化和特征组合增强BERT的答案选择模型

Answer selection model based on pooling and feature combination enhanced BERT

当前主流模型无法充分地表示问答对的语义,未充分考虑问答对主题信息间的联系并且激活函数存在软饱和的问题,而这些会影响模型的整体性能。针对这些问题,提出了一种基于池化和特征组合增强BERT的答案选择模型。首先,在预训练模型BERT的基础上增加对抗样本并引入池化操作来表示问答对的语义;其次,引入主题信息特征组合来加强问答对主题信息间的联系;最后,改进隐藏层的激活函数,并用拼接向量通过隐藏层和分类器完成答案选择任务。在SemEval-2016CQA和SemEval-2017CQA数据集上进行的验证结果表明,所提模型与tBERT模型相比,准确率分别提高了3.1个百分点和2.2个百分点;F1值分别提高了2.0个百分点和3.1个百分点。可见,所提模型在答案选择任务上的综合效果得到了有效提升,准确率和F1值均优于对比模型。

Current main stream models cannot fully express the semantics of question and answer pairs,do not fully consider the relationships between the topic information of question and answer pairs,and the activation function has the problem of soft saturation,which affect the overall performance of the model.To solve these problems,an answer selection model based on pooling and feature combination enhanced BERT(Bi-directional Encoder Representations from Transformers)was proposed.Firstly,adversarial samples and pooling operation were introduced to represent the semantics of question and answer pairs based on the pre-training model BERT.Secondly,the relationships between topic information of question and answer pairs were strengthened by the feature combination of topic information.Finally,the activation function in the hidden layer was improved,and the splicing vector was used to complete the answer selection task through the hidden layer and classifier.Model validation was performed on datasets SemEval-2016CQA and SemEval-2017CQA.The results show that compared with tBERT model,the proposed model has the accuracy increased by 3.1 percentage points and 2.2 percentage points respectively,F1 score increased by 2.0 percentage points and 3.1 percentage points respectively.It can be seen that the comprehensive effect of the proposed model on the answer selection task is effectively improved,and both of the accuracy and F1 score of the model are better than those of the model for comparison.