基于LSTM与衰减自注意力的答案选择模型

Answer selection model based on LSTM and decay self-attention

针对答案选择过程中存在语句特征、语句间的相关语义信息提取不充分的问题,在长短时记忆网络(LSTM)的基础上,提出基于LSTM和衰减自注意力的答案选择模型(DALSTM).DALSTM使用LSTM和衰减自注意力编码层提取丰富的上下文语义信息,通过衰减矩阵缓解反复使用注意力机制出现的权重过集中于关键词的问题.使用注意力机制对问题与答案间的信息进行双向交互,融合问答对间的相似性特征,丰富问答对间的相关语义信息.在WiKiQA、TrecQA及InsuranceQA数据集上的模型评估结果表明,相较于其他基于BiLSTM的先进模型,DALSTM的整体性能表现更好,3个数据集的平均倒数排名(MRR)分别达到0.757、0.871、0.743.

An answer selection model based on the long short-term memory(LSTM)and decay self-attention(DALSTM)was proposed on the basis of LSTM network,aiming at the problem of insufficient extraction of sentence features and related semantic information between sentences in the answer selection process.Contextual semantic information was extracted more fully by DALSTM which used LSTM and decay self-attention coding layer,and the problem of weight over-focused on keywords caused by repeated use of the attention mechanism was alleviated by the delay matrix.The attention mechanism was used to conduct bidirectional interaction between the information of question and answer,integrate the similarity features between question and answer pairs,and enrich the relevant semantic information between question and answer pairs.DALSTM was evaluated on WiKiQA,TrecQA,and InsuranceQA data sets.Evaluation results showed that compared with other advanced BiLSTM-based models,the DALSTM model had a better overall performance,mean reciprocal rank(MRR)of three data sets reached 0.757,0.871 and 0.743,respectively.