多头自注意力机制Siamese网络文本相似度计算方法

Siamese network text similarity calculation with multi-head self-attention mechanism

文本相似度的计算是自然语言处理的核心问题.现有的文本相似度计算方法,存在对于深层次的语义信息提取的不充分,且对长文本的相似度计算能力有限的问题.针对现有文本相似度计算方法的缺陷,提出一种基于多头自注意力机制的Siamese网络,利用双向GRU为基础的Siamese模型精确提取文本样本中上下文的语义信息,同时加入多头自注意力机制学习长文本深层次的语义信息.在公开的SICK数据集上,实验结果表明加入多头自注意力机制的Bi-GRU Siamese网络模型可以学习到长文本深层次的语义信息,对比其他的文本相似度的计算方法,相关系数显著提升,处理长文本效果较好.

Calculating text similarity is the key to natural language processing.Existing text similarity calculation methods can not fully extract deep semantic information.And the ability to calculate the similarity of long texts is limited.Aiming at the shortcoming of existing text similarity calculation methods,a Siamese network based on multi-head self-attention mechanism is proposed.This method uses the Siamese model based on bidirectional GRU to accurately extract the semantic information of the context.Then learning deep semantic information of long text by adding multi head self-attention mechanism.Experimental results show that Siamese network with multi-head self-attention mechanism can learn the deep semantic information of long text in the SICK dataset.Compared with other text similarity calculation methods,the correlation coefficient of the proposed method is significantly improved.The effect of processing long text is improved.