基于多模态数据的学习投入评估方法分析

Analysis of Learning Engagement Assessment Methods Based on Multimodal Data

学习投入是学生学习表现的关键影响因素,学习投入的自动识别是被广泛研究的问题。随着信息与传感技术的发展,在真实课堂中采集学生的多模态数据成为可能,如何利用多模态数据提升学习投入识别的准确率是值得研究的问题。文章分析基于外显或内隐信息的单模态模型和基于早期、晚期、混合融合方法的多模态模型,探究:(1)多模态数据相比于单模态数据在识别学习投入状态上的优势;(2)多模态数据融合方法对学习投入状态评估的影响。结果表明:结合外显与内隐信息可以更准确地识别学习投入状态;基于早期或混合融合方式的多模态模型在学习投入状态识别任务上有更佳的表现;融合方式对于最终结果准确性非常重要,不当的融合方式可能引入噪声,降低模型表现。

Learning engagement is a key factor influencing students'learning performance,and the automatic identification of learning engagement has been widely studied.With the development of information and sensing technology,it has become possible to collect multimodal data of students in real classrooms,and how to use multimodal data to improve the accuracy of learning engagement identification is a problem worthy of research.This paper analyzes the unimodal model based on explicit or implicit information and the multimodal model based on early,late,and hybrid fusion methods,and explores both the advantages of multimodal data compared with unimodal data in identifying learning engagement states and the impact of multimodal data fusion method on learning engagement state assessment.The results show that:the combination of explicit and implicit information can identify learning engagement state more accurately;Multimodal models based on early or hybrid fusion approaches have better performance on learning engagement state identification tasks;The fusion method is important for the accuracy of the final results,and an inappropriate fusion method may introduce noise and degrade the performance of the model.