基于块注意力机制和Involution的文本情感分析模型

Analyzing Text Sentiments Based on Patch Attention and Involution

【目的】解决卷积核宽度与词向量维度相同使得卷积层参数过多的问题,解决卷积操作的稀疏连接以及卷积的空间不变性和通道特异性不适用于文本任务的问题。【方法】提出一种基于块注意力机制和Involution的文本情感分析模型。模型先对分词后的单个词向量进行变形,将一维词向量变形为n×n词矩阵块,然后将句子中多个词的词矩阵块拼接成句子矩阵。句子矩阵经过块注意力机制层,增强了文本特征的上下文相关性及位序信息,再通过采用具有空间特异性和通道不变性的Involution对句子矩阵进行特征提取,最后使用全连接层进行文本情感分类。【结果】在三个文本情感分析公开数据集waimai_10k、IMDB、Tweet上的实验表明,所提模型的分类准确率分别达到88.47%、86.22%、94.42%,与词向量卷积网络和循环神经网络中的Bi-LSTM模型相比准确率分别提高6.47、7.72、9.35个百分点和1.07、1.01、0.59个百分点。【局限】所提模型在大型数据集上的分类准确度低于中小型数据集。【结论】引入块注意力机制和Involution的文本情感分析模型解决了参数量过多、卷积操作的稀疏连接以及卷积的空间不变性和通道特异性的问题,在不同数据集上,与传统卷积模型比较,本文模型的准确率有所提升。

[Objective]Once the width of the convolution kernel is the same as the dimension of the word vector,the convolution layer will have too many parameters.The sparse connection of convolution operation,the spatial invariance,and the channel specificity of convolution are not suitable for text tasks.This paper will address these issues.[Methods]We proposed a sentiment analysis model for texts based on patch attention mechanism and Involution.The model first transformed the single-word vector after word segmentation and transformed the onedimensional word vector into n×n word matrix blocks.Then,we spliced the word matrix blocks of multiple words in the sentence into a sentence matrix.Third,the patch attention mechanism layer enhanced the sentence matrix’s context relevance and position order information of text features.Fourth,we used the involution with spatial specificity and channel invariance to extract the sentence matrix features.Finally,we used the full connection layer for text sentiment classification.[Results]We examined the proposed model with three public data sets waimai_10k,IMDB,and Tweet.Its classification precision reached 88.47%,86.22%,and 94.42%,respectively,which were 6.47%,7.72%,9.35%and 1.07%,1.01%,0.59%higher than Bi-LSTM model in word vector convolution network and recurrent neural network.[Limitations]The classification accuracy of this model on large datasets is not as high as on small and medium-sized datasets.[Conclusions]The proposed model solves the problems of excessive parameters,sparse connection of convolution operation,spatial invariance,and channel specificity of convolution,which yield better performance than the traditional convolution models.