基于Inception V3的高校学生课堂行为识别研究

Research on Action Recognition of Student Classroom Behavior in Universities Based on Inception V3

随着人工智能和深度学习在教育领域的交叉融合,行为识别技术为学生课堂行为观察提供了一种有别于传统的新方法。以云南省X高校课堂视频为基础,经过预处理,获得六大类行为(听课、看书、书写、拍照、低头玩手机、桌面玩手机)30000张图像样本,运用Inception V3算法模型进行了研究,实验结果:六大类行为总识别率达到88.10%,但各个行为识别率有所不同,其中“拍照”和“听课”识别率较高。通过进一步的混淆矩阵分析,得到结论:模型对动作姿态单一的行为特征提取效果较好,但模型对手机、笔、课本等重要用具不够重视,不能识别书写动作和眼神角度,导致“看书”“书写”“低头玩手机”和“桌面玩手机”行为因人体动作姿态相似容易混淆。

With the cross-integration of AI and deep learning in the field of education,action recognition provides a new method for student classroom behavior observation,which is different from traditional method.Based on classroom video in X university of Yun⁃nan province,this paper collects the original data by shooting students'class video.After preprocessing,the dataset of 30000 sam⁃ples of six categories of behavior(watch,read,note,picture,eye-down,phone-desk)are obtained.And finally,action recognition of classroom behavior is preliminarily studied by using Inception V3 CNN model.Result:the total recognition rate of six categories of behavior is 88.10%,but the recognition rate of each behavior is different,"picture"and"watch"behavior are higher,other be⁃havior are lower.Through further analysis of confusion matrix and error recognition samples,conclusion is drawn:The model has a higher recognition rate of simple action posture,behavior features extracted from deep learning are better.However,the model does not attach enough importance to the important props like phone,pen and book,it also can not recognize the"writing action"and"eye angle"very well,which leads to the confusion of"read","note","eye-down",and"phone-desk"because of the similarity of action posture.