基于自注意力机制与特征融合的课堂学生表情识别模型

Classroom Student Expression Recognition Model Based on Self Attention Mechanism and Feature Fusion

为解决通常课堂场景下学生人脸表情识别的遮挡问题,通过部分分割和随机遮挡策略将原图分割成多路人脸图像,采用相同的残差网络提取特征,借助自注意力机制为多路网络分配不同权重,再对损失函数进行约束以限制遮挡支路权重始终小于眼部支路权重,进而得到加权后的支路特征并通过特征融合形成全局特征。在公开数据集FERplus上实验表明,模型能够大幅度提升人脸表情识别的准确率,有效缓解复杂场景下因遮挡造成的信息损失问题。

In order to solve the occlusion problem of students’ facial expression recognition in common class-room scenes, the original image is divided into multiple face images through partial segmentation and random occlusion strategies, the same residual network is used to extract features, the self attention mechanism is used to assign different weights to the multiple networks, and then the loss function is constrained to limit the weight of the occlusion branch to always be less than the weight of the eye branch. Then, the weighted branch features are obtained and global features are formed through feature fusion. Experiments on the public dataset FERplus have shown that the model can significantly improve the accuracy of facial expression recognition and effectively alleviate the problem of information loss caused by occlusion in complex scenes.