基于多模态特征融合的抑郁症识别

Feature-level Multimodal Fusion for Depression Recognition

抑郁症是一种常见的精神疾病,现有的抑郁症诊断主要依赖于抑郁量表和精神科医生的访谈,具有较强的主观性。近年来,越来越多的研究者致力于通过脑电特征或音频特征识别抑郁症患者,但并未有研究将脑电信息与音频信息有效地结合起来,忽略了音频和脑电数据之间的相关性。因此本文提出一种基于全连接神经网络的多模态特征融合模型,通过对音频模态和脑电模态信息的特征融合提升抑郁症识别的准确率,为抑郁症的识别提供新的角度和方法。实验表明,多模态特征融合在MODMA数据集上的抑郁症识别准确率达到了81.58%且高于单模态抑郁症识别方法的准确率。这表明,相比于单模态识别,多模态特征融合模型能够提高抑郁症识别的准确率。

Depression is a common psychiatric disorder.However,the existing diagnostic methods for depression mainly rely on scales and interviews with psychiatrists,which are highly subjective.In recent years,researchers have devoted themselves to identifying depressed patients by EEG features or audio features,but no study has effectively combined EEG information with au-dio information,ignoring the correlation between audio and EEG data.Therefore,this study proposes a feature-level multimodal fusion model to improve the accuracy of depression recognition.We combine the audio and EEG modality information based on a fully connected neural network.Our experiments show that the accuracy of depression recognition using feature-level multimodal fusion model on the MODMA dataset reaches 81.58%,which is higher than that of using single-modality.The results indicate that the feature-level multimodal fusion model can improve the accuracy of depression recognition compared to single-modality.Our research provides a new perspective and method for depression recognition.