摘要
随着多媒体的快速发展,单纯采用文本的方面类情感分析,不能准确识别用户所表达的情感。而现有图文数据的方面类情感分析方法仅考虑图文模态间的交互,忽略图文数据的不一致性和相关性。因此,提出联合方面注意力交互网络(JAAIN)模型的图文方面类情感识别方法。所提方法针对图文数据的不一致性与相关性,通过多层次融合方面信息和图文信息,去除与给定方面无关的文本和图像,增强给定方面的图文模态数据的情感表示,将文本数据情感表示、图像数据情感表示及方面类情感表示进行拼接融合与全连接,实现图文方面类情感判别。在数据集Multi-ZOL上进行实验,实验结果表明:所提模型能够提升图文方面类情感判别的性能。
Due to the quick development of social media,the sentiment conveyed by users cannot be reliably identified by an Aspect-Category Sentiment Analysis of the text alone.However,the existing Aspect-Category Sentiment Analysis methods for image and text data only consider the interaction between image and text modalities,ignoring the inconsistency and correlation of image and text data.Therefore,this paper proposes a joint aspect attention interaction network(JAAIN)model for aspect-category sentiment identification.The suggested technique improves the representation of image and text modalities in particular aspects by multi-level aspect,image,and text information fusion.It does this by removing the text and images that are unrelated to certain aspects.The text data sentiment representation,image data sentiment representation and aspect category sentiment representation are concatenated,fused and fully connected to realize sentiment discrimination of image and text aspects.The experimental results show that the proposed model can improve the performance of sentiment identification in images and text on the Multi-ZOL Dataset.
作者
赵一成
王素格
廖健
何东欢
ZHAO Yicheng;WANG Suge;LIAO Jian;HE Donghuan(School of Computer&Information Technology,Shanxi University,Taiyuan 030006,China;Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education,Shanxi University,Taiyuan 030006,China)
出处
《北京航空航天大学学报》
EI
CAS
CSCD
北大核心
2024年第2期569-578,共10页
Journal of Beijing University of Aeronautics and Astronautics
基金
国家自然科学基金(62076158,61906112)
山西省太原市小店区科技局项目(2020XDCXY05)。
关键词
方面类情感分析
注意力机制
多模态情感分析
情感表示
多模态融合
aspect-category sentiment analysis
attention mechanism
multi-modal sentiment analysis
sentiment expression
multi-modal fusion