联合方面注意力交互的图文方面类情感识别

Image-text aspect emotion recognition based on joint aspect attention interaction

随着多媒体的快速发展,单纯采用文本的方面类情感分析,不能准确识别用户所表达的情感。而现有图文数据的方面类情感分析方法仅考虑图文模态间的交互,忽略图文数据的不一致性和相关性。因此,提出联合方面注意力交互网络(JAAIN)模型的图文方面类情感识别方法。所提方法针对图文数据的不一致性与相关性,通过多层次融合方面信息和图文信息,去除与给定方面无关的文本和图像,增强给定方面的图文模态数据的情感表示,将文本数据情感表示、图像数据情感表示及方面类情感表示进行拼接融合与全连接,实现图文方面类情感判别。在数据集Multi-ZOL上进行实验,实验结果表明:所提模型能够提升图文方面类情感判别的性能。

Due to the quick development of social media,the sentiment conveyed by users cannot be reliably identified by an Aspect-Category Sentiment Analysis of the text alone.However,the existing Aspect-Category Sentiment Analysis methods for image and text data only consider the interaction between image and text modalities,ignoring the inconsistency and correlation of image and text data.Therefore,this paper proposes a joint aspect attention interaction network(JAAIN)model for aspect-category sentiment identification.The suggested technique improves the representation of image and text modalities in particular aspects by multi-level aspect,image,and text information fusion.It does this by removing the text and images that are unrelated to certain aspects.The text data sentiment representation,image data sentiment representation and aspect category sentiment representation are concatenated,fused and fully connected to realize sentiment discrimination of image and text aspects.The experimental results show that the proposed model can improve the performance of sentiment identification in images and text on the Multi-ZOL Dataset.