加强融合表情和语音的抑郁症检测模型

Depression detection model that enhances fusion of facial expressions and speech

抑郁症患者的表情和语音具有直观、易于获取等优点,已被广泛应用于抑郁症检测,但现有研究存在忽略表情变化过程包含的信息在抑郁症检测中的作用,未能将动态表情包含的信息与静态表情、语音有效结合,识别准确度不高等问题。针对上述问题,提出一种用动态表情和语音加强融合静态表情特征的抑郁症检测模型。在语音特征提取模块中加入Bi-LSTM网络,挖掘语音的时序信息,用情感语音迁移学习,再用抑郁症语音训练。表情特征提取模块采用双通道结构,利用混合注意力机制分别提取动态表情和静态表情特征,特征更具判别性。特征加强融合模块用语音和动态表情加强融合静态表情,特征信息互补加强。实验结果表明,所提方法在AVEC2014数据集上检测的RMSE和MAE降低到8.21和6.03,优于目前使用语音和表情检测抑郁症的方法。

The expressions and speech of patients with depression have the advantages of being intuitive and easy to be obtained,so they have been widely used in the depression detection.However,the existing research has overlooked the role of the information contained in the process of expression change in the depression detection,and has failed to effectively combine the information contained in dynamic expressions with static expressions and speech,which results in low recognition accuracy.In view of the above,a depression detection model enhancing the fusion of static expression features with dynamic expressions and speech is proposed.The Bi-LSTM network is added to the speech feature extraction module to mine the temporal information of speech,and perform transfer learning with emotional speech,and then implement training with depression speech.The expression feature extraction module is structured with a dual channel,and a mixed attention mechanism is utilized to extract dynamic and static expression features,so the features are more discriminative.In the feature enhancement fusion module,the speech and dynamic expressions are used to enhance the fusion of static expressions,which enhances the complementation of feature information.The experiment results show that the proposed method can reduce the RMSE(root mean square error)and MAE(mean absolute error)detected on the dataset AVEC2014 to 8.21 and 6.03,respectively,so it is superior to the current methods that detect depression with speech and facial expressions.