基于共享词嵌入空间的跨语言相似问句挖掘

Cross-Language Similar Questions Mining Based on Shared Word Embeddings Space

针对跨语言相似问句查找问题,提出一种基于共享词嵌入空间计算中文和英文问句相似性的方法。该方法首先用fastText训练中、英文词嵌入,之后训练中文词嵌入转换到英文词嵌入的线性矩阵,再对待处理的中、英文问句做相应处理,生成英文空间下句子嵌入,根据句子嵌入余弦相似性计算句子相似性。实验结果表明该方法是可行的。

Aiming at the problem of cross-language similar questions lookup, proposes a method based on Sentence2Embeddings to calculate the similarity between Chinese and English questions. This method first trains Chinese and English word embeddings with fastText, and then trains the Chinese word embeddings to convert to the linear matrix of English word embeddings, then deals with the Chinese questions and English questions sentence to be processed, and generates sentence embedding in English space. Sentence similarity is calculated based on sentence embedding. Experimental results show that the method is feasible.