基于主辅网络特征融合的语音情感识别

Feature Fusion Based on Main-Auxiliary Network for Speech Emotion Recognition

为了有效特征提取与融合提高语音情感识别率,提出了一种使用主辅网络进行深度特征融合的语音情感识别算法。首先将段特征输入BLSTM-Attention网络作为主网络,其中注意力机制能够关注语音信号中的情感信息;然后,把Mel语谱图输入CNN-GAP网络作为辅助网络,GAP可以减轻全连接层带来的过拟合;最后,将两个网络提取的深度特征以主辅网络方式进行特征融合,解决不同类型特征直接融合带来的识别结果不理想的问题。在IEMOCAP数据集上对比4种模型的实验结果表明,使用主辅网络深度特征融合的WA和UA均有不同程度的提高。

Speech emotion recognition is an important research direction of human-computer interaction.Effective feature extraction and fusion are among the key factors to improve the rate of speech emotion recognition.In this paper,a speech emotion recognition algorithm using Main-auxiliary networks for deep feature fusion was proposed.First,segment features are input into BLSTM-attention network as the main network.The attention mechanism can pay attention to the emotion information in speech signals.Then,the Mel spectrum features are input into Convolutional Neural Networks-Global Average Pooling(GAP)as auxiliary network.GAP can reduce the overfitting brought by the fully connected layer.Finally,the two are combined in the form of Main-auxiliary networks to solve the problem of unsatisfactory recognition results caused by direct fusion of different types of features.The experimental results of comparing four models on IEMOCAP dataset show that WA and UA using the depth feature fusion of the Main-Auxiliary network are improved to different degrees.