基于多模态数据的课堂学生行为识别技术的探究

Research on Action Recognition Technology of Classroom Students Based on Multi-Modal Data

为了实现学生课堂行为如起立、端坐、举手的识别,提高识别的准确率和召回率,使用多模态数据如人体关键点信息、RGB图像等进行实验,并对训练数据的有效性进行探究,最终在测试集上达到92%的准确率和96%的召回率。实验结果表明,对多模态数据的合理应用以及对数据进行有效处理,可以有效提高识别任务的效果。

In order to realize the action recognition of classroom students,such as standing,sitting and raising hands,and improve the precision and recall rate of the recognition task,multi-modal data such as human body keypoints and RGB image are used in this paper,and the validity of training data is explored.Finally,the precision and recall rate reached 92%and 96%in the test set.The experimental results show that the proper application of multi-modal data and the effective processing of the data can effectively improve the effect of the recognition task.