基于多任务学习的课堂表情分类模型

Classroom Expression Classification Model Based on Multitask Learning

基于课堂视频图像理解的学生表情识别与学习情感分析技术已然成为当前智慧教育领域的研究热点,但在图像视频采集质量低、环境复杂、多目标遮挡严重的真实应用场景中往往面临很大的挑战。针对目前大多课堂表情分类模型仅关注离散表情单一维度存在的不足,提出了一种多任务识别的学生表情分类模型。首先,基于课堂表情分类模型构建了真实场景下的多任务课堂表情数据集,并通过数据平衡技术解决数据集类别标签分布不平衡问题。其次,提出了一种基于多任务学习的课堂表情分类模型,通过引入知识蒸馏并设计双通道融合机制,有效融合离散表情、人脸动作单元和效价-唤醒三个表情识别任务,利用多任务之间的关系进一步增强离散表情分类任务的性能。最后,该方法在多个数据集上与现有先进方法进行了实验对比分析,结果表明所提模型能够有效提升表情分类精度,且在课堂表情多任务识别中具有优越表现,为实现课堂情感多维度评估分析提供技术支持。

Facial expression recognition and learning sentiment analysis based on classroom video image understanding have become research hotspots in smart education.However,these applications often face great challenges in real-world scenarios with low-quality image and video acquisition,and serious multi-target occlusion in complex environments.In this paper,a multitask recognition model for classifying student expressions is proposed.Firstly,this study constructs a multitask classroom expression dataset and effectively alleviates the imbalance of class label distribution in the dataset.Secondly,a classroom expression classification model based on multitask learning is proposed.By introducing knowledge distillation and designing a dual-channel fusion mechanism,the model effectively integrates the three tasks of discrete expression recognition, facial action unit detection andvalence-arousal estimation. This integration leverages the relationship between multitasksto further enhance the performance of discrete expression classification. Finally, the proposedmethod is compared with the existing advanced methods across multiple datasets.Results show that the proposed model effectively improves the accuracy of expression classification,and demonstrates superior performance in the multitask recognition of classroomexpressions, which provides technical support for multi-dimensional evaluation and analysisof classroom emotions.