基于Transformer模型的问句语义相似度计算

Semantic similarity calculation of questions based on Transformer model

针对现有方法准确率不高、不能充分捕捉句子深层次语义特征的问题,提出一种基于Transformer编码器网络的问句相似度计算方法。在获取句子语义特征前引入交互注意力机制比较句子间词粒度的相似性,通过注意力矩阵和句子矩阵相互生成彼此注意力加权后的新的句子表示矩阵,将获取的新矩阵同原始矩阵拼接融合,丰富句子特征信息;将拼接后的句子特征矩阵作为Transformer编码器网络的输入,由Transformer编码器分别对其进行深层次语义编码,获得句子的全局语义特征;通过全连接网络和Softmax函数对特征进行权重调整,得到句子相似度。在中文医疗健康问句数据集上模型取得了90.2%的正确率,较对比模型提升了将近4.2%,验证了该方法可以有效提高句子的语义表示能力和语义相似度的准确性。

To solve the problem that the existing methods are not accurate and can not fully capture the deep semantic features of sentences,a question similarity calculation method based on transformer encoder network was proposed.Before acquiring sentence semantic features,interactive attention mechanism was introduced to compare the similarity of word granularity between sentences.The attention matrix and sentence matrix were used to generate a new sentence representation matrix weighted by each other’s attention.The new matrix was combined with the original matrix to enrich the sentence feature information.The spliced sentence feature matrix was used as the input of the transformer encoder network,and the transformer encoder encoded them in deep level to obtain the global semantic features of sentences.The feature weight was adjusted and the sentence similarity was calculated through the full connection network and Softmax function.On the data set of Chinese medical and health questions,the accuracy of the model is 90.2%,which is nearly 4.2%higher than that of the comparative model.It is verified that this method can effectively improve the semantic representation ability of sentences and the accuracy of semantic similarity.