基于交互式学习与多头注意力机制的金融文本情感分类

Financial Text Sentiment Classification Based on Interactive Learning and Attention Mechanism

针对现有的金融文本情感分类模型在对文本表示进行建模时忽略了上下文语义和情感倾向之间交互关系的问题,本文提出了基于交互式学习与多头注意力机制的金融文本情感分类模型(Interactive learn⁃ing and multi-head attention mechanism,ILMA)。该模型首先利用BERT模型来获得上下文和情感倾向信息的词嵌入表示;然后分别使用两个双向GRU(BiGRU)神经网络来学习上下文语义和情感倾向信息的随机依赖性;之后利用交互式学习机制和多头注意力机制来学习文本语境语义与情感倾向信息之间的交互表示;最后将生成的上下文语义和情感倾向信息的表示序列进行连接后,作为softmax分类器的输入进行分类。在四个公开的金融数据集上进行了实验,将ILMA模型与其他方法进行比较,结果表明ILMA模型的分类准确性相对于其他模型有了显著提升。

Aiming at the problem that the existing financial text emotion classification model ignores the interaction between context semantics and emotion tendency when modeling text representation.This paper proposes an interactive learning and multi head attention mechanism(ILMA)based on interactive learning and multi head attention mechanism.Firstly,the Bert model is used to obtain the word embedded representation of context and emotional tendency information,and then two two-way Grus are used respectively(BiGRU)neural network is used to learn the random dependence of context semantics and emotional orientation information;then interactive learning mechanism and multi head attention mechanism are used to learn the interactive representa⁃tion between text context semantics and emotional orientation information;finally,the generated representation sequence of context semantics and emotional orientation information is connected and classified as the input of softmax classifier.Experiments are car⁃ried out on four public financial data sets,and the ILMA model is compared with other methods.The results show that the classifi⁃cation accuracy of ILMA model is significantly improved compared with other models.