多模态的情感分析技术综述

Summary of Multi-modal Sentiment Analysis Technology

情感分析是指利用计算机自动分析确定人们所要表达的情感,其在人机交互和刑侦破案等领域都能发挥重大作用。深度学习和传统特征提取算法的进步为利用多种模态进行情感分析提供了条件。结合多种模态进行情感分析可以弥补单模态情感分析的不稳定性以及局限性等缺点,能够有效提高准确度。近年来,研究者多用面部表情信息、文本信息以及语音信息三种模态进行情感分析。主要从这三种模态对多模态情感分析技术进行综述:首先对多模态情感分析的基本概念以及研究现状进行简要介绍;其次总结了常用的多模态情感分析数据集;然后分别对现有的基于面部表情信息、文本信息和语音信息的单模态情感分析技术进行简要叙述;接下来详细介绍了模态融合技术,并依据不同的模态融合方式对多模态情感分析技术的现有成果进行重点描述;最后讨论了多模态情感分析存在的问题以及未来的发展方向。

Sentiment analysis refers to the use of computers to automatically analyze and determine the emotions that people want to express.It can play a significant role in human-computer interaction and criminal investigation and solving cases.The advancement of deep learning and traditional feature extraction algorithms provides conditions for the use of multiple modalities for sentiment analysis.Combining multiple modalities for sentiment analysis can make up for the instability and limitations of single-modal sentiment analysis,and can effectively improve accuracy In recent years,researchers have used three modalities of facial expression information,text information,and voice information to perform sentiment analysis.This paper mainly summarizes the multi-modal sentiment analysis technology from these three modalities.Firstly,it briefly introduces the basic concepts and research status of multimodal sentiment analysis.Secondly,it summarizes the commonly used multi-modal sentiment analysis datasets.It gives a brief description of the existing single-modal emotion analysis technology based on facial expression information,text information and voice information.Next,the modal fusion technology is introduced in detail,and the existing results of the multi-modal sentiment analysis technology are mainly described according to different modal fusion methods.Finally,it discusses the problems of multi-modal sentiment analysis and future development direction.