基于自注意力和门控机制的答案选择模型

Answer selection model based on self-attention and gating mechanism

针对现有答案选择方法语义特征提取不充分和准确性差的问题,引入自注意力和门控机制,提出了一种答案选择模型。该模型首先在问题和答案文本内部利用层叠自注意力进行向量表示,并在自注意力模块中让单词和位置分开进行多头注意力;然后将答案句通过卷积神经网络(Convolutional neural network, CNN)得到的向量表示输入注意力层,根据问题生成与问题相关的答案表示,并通过门控机制融合两种表示;最后计算问题和答案文本的相关性分数,得到候选答案的排名和标注。结果表明:该模型与双向长短时记忆网络模型、自注意力模型和基于注意力的双向长短时记忆网络模型相比,在WebMedQA数据集上平均倒数排名分数分别提高了8.37%、4.79%和2.03%,预测答案正确率也有提高。这表明提出的模型能够捕获更丰富的语义信息,有效提升了答案选择的性能。

In view of the problem of insufficient semantic feature extraction and poor accuracy of existing answer selection methods, self-attention and gating mechanism are introduced to propose an answer selection model. In the question and answer texts, this model firstly adopted cascading self-attention for vector representation, and then separated the words and positions in the self-attention module for multi-head attention. After that, the answer sentences were input into the attention layer, through the vector representations obtained through the convolutional neural network(CNN). The answer representation related to the question was generated based on the question, and the two representations were merged through gating mechanism. Finally, the correlation score between question and answer texts was calculated to obtain the ranking and labeling of the candidate answer. The experimental results show that compared with the bidrectional long short-term memory(BiLSTM) model, the self-attention model and the attention-based BiLSTM model, the mean reciprocal rank scores on the WebMedQA dataset grew by 8.37%, 4.79% and 2.03% respectively, and the predicted accuracy of answers was also improved, suggesting that the proposed model can capture more abundant semantic information and effectively improve the performance of answer selection.