基于多通道的语义信息融合交互方法

Semantic information fusion interaction method based on multi-channel

提出一种基于多通道的语义信息融合交互方法,使用不同的网络结构来提取原始语音信息、图像信息以及行为信息的语义特征,通过隐马尔可夫模型加强不同特征之间的交互,使用注意力机制建立语义信息融合,捕获了深层语义特征。在IEMOCAP数据集上验证了所提方法的有效性。

Therefore,in order to solve the existing problem of single interaction channel,this paper proposes a semantic information fusion interaction method based on multi-channel,using different network structures to extract the semantic features of original voice information,image information and behavioral information,through hidden Markov the model strengthens the interaction between different features,uses the attention mechanism to establish semantic information fusion,and captures deep semantic features.The effectiveness of the method proposed in this article is verified on the IEMOCAP data set.