融合多头注意力机制的多任务情感分类研究

Researoh on multi-task sentiment classification based on the multi-head attention mechanism

目的:针对当前多任务学习(multi-task learning,MTL)方法往往忽视子任务之间特征相关性的问题,提出一种融合多头注意力机制(multi-head attention,MHA)的多任务情感分类(MHA-MTL)方法。方法:首先采用MHA提取文本重要特征,在训练中对多领域数据集进行动态的自动归类。文中构造了基于长短期记忆网络(long short-term memory,LSTM)和逐点卷积网络的多任务情感分类器,并且设计了辅助分类任务的损失函数,在模型整体训练中,动态优化特征提取和分类器参数。结果:在五个领域的中文任务集上进行实验,实验结果表明与单任务和传统的多任务学习方法相比,本文提出的方法在准确性和F_(1)值上都有明显的提升。结论:使用融合多头注意力机制的多任务学习方法能够有效提升情感分类模型的性能。

Aims:Aiming at the problem that current multi-task learning(MTL)methods often ignore the feature correlation among subtasks,a multi-head attention based multi-task sentiment classification(MHA-MTL)method was proposed.Methods:Firstly,MHA was used to extract important features of text;and the dataset was dynamically and automatically classified during training.In this paper,the multi-task sentiment classifier based on long short-term memory(LSTM)and point-by-point convolutional network was constructed;and the loss function of the auxiliary classification task was designed.In the overall model training,the feature extraction and classifier parameters were dynamically optimized.Results:Experimental results on Chinese task sets in five domains showed that the proposed method had significant improvement in accuracy and F_(1) value compared with single-task and traditional multi-task learning methods.Conclusions:The multi-task learning method combined with multi-attention mechanism can effectively improve the performance of sentiment classification models.