问答系统中问句相似度研究

Study on the Similarity of Question Sentences in Question and Answer System

百度知道中用户提出问题较短,采用常规基于空间向量的TF-IDF句子相似度计算、基于语义依存关系的句子相似度计算等方法往往很难较好完成其相似度计算。鉴于此,基于长度较短问句的特点,引入问题元和词模思想,对用户问题进行分解,并与传统相似度计算方法相融合,提出新的相似度计算方法。对于长度低于20个词的问句,与传统TF-IDF方法相比,F1值提高了12%。

In view of the short length of questions raised by Baidu users,the conventional space vector-based TF-IDF sentence similarity calculation and the semantic similarity-based sentence similarity calculation are often difficult to perform good similarity calculation.To this end,this paper introduces the idea of problem element and lexical model for the characteristics of short-length question,decomposes the users’problems and then combines with the traditional similarity calculation method,and proposes a new similarity calculation method.For questions with a length of less than 20 words,the F1 value is increased by 12%compared to the traditional TF-IDF method.