带有GRU单元的通用句嵌入算法研究

Algorithm of Sentence Embedding with GRU Cells

在自然语言处理领域,将词语和句子转换成定长稠密向量的算法被称为词嵌入和句嵌入算法。句嵌入算法的主要目的是寻找通用的嵌入表达,使该表达中包含句子的特征,蕴含句子语义情感等信息,并能够在自然语言处理中的其他下游任务中作为输入值使用。目前主流的句嵌入算法多为有监督的方法,能在一些具体的自然语言处理任务中提供很好的表现,但是通用性不足,而且需要针对具体的任务寻找不同的标签语料库,对语料库的要求较高。而无监督的方法能够很好地解决这些问题,因此使用无监督的RNN编码解码器模型去训练更具有通用性的句嵌入模型。并使用固定窗口大小的句子上下文信息来构建损失函数,从而使向量表达更为准确。

In natural language processing area, the algorithms to turning words and sentences into length fixed, dense vectors are called word embedding and sentence embedding which are also called as word representation and sentence representation. Then main goal of sentence embed? ding algorithm is to find a common representation which contains the sentence's feature like semantics and sentiment and can be used in other downstream tasks in NLP. From now on, most of sentence embedding algorithm are supervised methods, proved useful in some specific tasks like sentiment analysis, but lack of commonality and well-labeled corpus. Unsupervised methods can overcome these drawbacks. Tries to use RNN encoder-decoder model to train sentence embedding, uses fixed window of context to make it more accurate.