基于集成学习与聚类联合标注的多模态个体情绪识别

Multimodal individual emotion recognition with joint labeling based on integrated learning and clustering

针对通用情绪识别模型面对不同个体时的低识别精度问题,提出一种基于集成学习与聚类联合标注的多模态个体情绪识别方法。该方法首先基于公共数据集训练通用情绪识别模型,然后分析公共数据集数据与个体无标签数据的分布差异,建立跨域模型来预测和标注个体数据的伪标签。同时,对个体数据进行加权聚类并标注聚类标签,利用聚类标签与伪标签进行联合标注,筛选高置信度样本进一步训练通用模型,得到个性化情绪识别模型。实验采集3名被试的3种情绪数据并使用该方法标注,最后优化得到的个性化模型对3种情绪的平均识别精度达到80%以上,相比原通用模型,至少提升了35%。

To address the low recognition accuracy of generic emotion recognition models when faced with different individuals,a multimodal individual emotion recognition technique based on joint labelling with integrated learning and clustering was proposed.The method first trained a generic emotion recognition model based on a public dataset,then anallysed the distributional differences between the data in the public dataset and the unlabelled data of individuals,and established a cross-domain model for predicting and labelling pseudo-labels of individual data.At the same time,the individual data were weighted clustered and labelled with cluster labels,and the cluster labels were used to jointly label with pseudolabels,and high confidence samples were screened to further train the generic model to obtain a personalized emotion recognition model.Using this method to annotate these data with the experimentally collected data of 3 emotions from 3 subjects,the final optimized personalized model achieved an average recognition accuracy of more than 80%for the 3 emotions,which was at least a 35%improvement compared to the original generic model.