基于主动样本精选与跨模态语义挖掘的图像情感分析

Image sentiment analysis via active sample refinement and cross-modal semantics mining

图像情感分析是机器视觉领域的研究热点,它面临的关键问题是:标注者的主观差异导致情感标签明确的高质量样本匮乏,且异构图像特征间跨模态语义未有效利用.为此,提出基于主动样本精选与跨模态语义挖掘的图像情感分析模型ASRF^(2)(active sample refinement&feature fusion):融合主动学习与样本精选思想,设计主动样本精选策略,优选情感标签明确的样本;对异构图像特征执行判别相关分析,生成能准确刻画图像情感内容的低维跨模态语义;采用跨模态语义训练Catboost模型,实现图像情感分析.在Twitter I与FI数据集上验证ASRF^(2)模型,识别准确率分别达90.06%和75.77%,优于主流基线且实时效率良好.与基线相比,ASRF^(2)模型仅需两类特征,参数调制简单,更易复现.ASR策略还具备一定的泛化性,可为基线模型提供优质训练样本,以改善识别性能.

Image sentiment analysis is a research focus in the field of computer vision.However,we are faced with the following key problems:First,owing to the subjective differences of different annotators,high-quality samples with definite sentimental annotations are very scarce.Second,the implicit cross-modal semantics among heterogeneous features has not been fully explored.To address these two problems,we propose an active sample refinement&feature fusion(ASRF^(2))via active sample refinement and cross-modal semantics mining:an active sample refinement strategy is designed by fusing the active learning and sample refinement ideas.High-quality samples with definite sentimental annotations are obtained in turn.Then,the state-of-the-art discriminant correlation analysis(DCA)algorithm is employed to fully mine the cross-modal correlations among the heterogeneous features.Low-dimensional but more discriminant cross-modal semantics that can better depict the key sentimental contents of images are generated.The cross-modal semantics is used to train a Catboost classifier and complete image sentiment analysis.We validate the proposed ASRF^(2)model on the Twitter I and FI datasets.The corresponding accuracies reach about 90.06%and 75.77%,respectively,which outperform other state-of-the-art baselines as well as the real-time efficiency.Compared with the baselines,the proposed model only needs two image features,and it is easy to tune and reproduced the ASRF^(2)model.Moreover,the ASR strategy is robust,which can offer many more high-quality samples for the baselines to improve the final recognition performance.