结合LDA与Word2vec的文本语义增强方法

Text Semantic Enhancement Method Combining LDA and Word2vec

文本的语义表示是自然语言处理和机器学习领域的研究难点,针对目前文本表示中的语义缺失问题,基于LDA主题模型和Word2vec模型,提出一种新的文本语义增强方法Sem2vec(semantic to vector)模型。该模型利用LDA主题模型获得单词的主题分布,计算单词与其上下文词的主题相似度,作为主题语义信息融入到词向量中,代替one-hot向量输入至Sem2vec模型,在最大化对数似然目标函数约束下,训练Sem2vec模型的最优参数,最终输出增强的语义词向量表示,并进一步得到文本的语义增强表示。在不同数据集上的实验结果表明,相比其他经典模型,Sem2vec模型的语义词向量之间的语义相似度计算更为准确。另外,根据Sem2vec模型得到的文本语义向量,在多种文本分类算法上的分类结果,较其他经典模型可以提升0.58%~3.5%,同时也提升了时间性能。

Text semantic representation is one of the most difficulty problems in natural language processing and machine learning.To solve the problem of semantic loss in text representation,this paper proposes a new text semantic representation method named Sem2vec(semantic to vector)model which is based on the LDA topic model and the Word2vec model.The topic similarity is calculated according to the word topic distribution obtained by the LDA model.Then the topic semantic word vectors are inputted into the Sem2vec model instead of the one-hot vector.Constrained by maximizing log-likelihood objective function,the parameters of the Sem2vec model are optimized.Finally,the semantic word vectors are learned by the Sem2vec model and the semantic representation of the text is further obtained.The experimental results on different datasets show that compared with the other classic models,the Sem2vec model is more accurate in calculating semantic similarity between words.Moreover,in different classification algorithms,the text semantic vectors generated by the Sem2vec model can improve the text classification results by 0.58%~3.5%and promote the time performance compared with the other classic models.