多模态数据下的学习投入特征画像研究

Study on Learning Engagement Feature Portrait Based on the Multi-modal Data

学习投入是教师研判学习活动的重要指标,为后续教学诊断奠定了基础。随着AI与教学深度融合不断推进,各种教学数据媒介日益增长,使多模态数据表征学习投入画像逐步成为教学领域的技术趋势。针对以往研究中数据高低特征互补性不足的现状,提出一种融合文本数据与图像数据的多模态学习投入特征画像构建方法,提升了多维学习投入的表征效果。具体的,采用transformer模型提取文本语义特征,联合FasterRCNN模型提取视觉特征,通过Attention特征融合得到不同维度学习投入的预测分类结果,最后借助可视化技术呈现学习投入画像。研究表明,该模型有效表征了学习群体在各维度的不均衡投入状态,有助于开展教学诊断与协作分组,有效辅助教师教学诊断,提高了协作分组活动的教学效率。

Learning engagement is an important indicator for teachers to study and judge learning activities,which lays a foundation for subsequent teaching diagnosis.With the continuous promotion of the deep integration of AI and teaching,various teaching data media are growing day by day,making multimodal data representation of learning engagement portrait gradually become the technical trend in the teaching field.In view of the lack of complementarities between high and low features of data in previous studies,the research proposes a construction method of multimodal learning engagement feature portrait that integrates text data and image data to improve the representation effect of multi-dimensional learning engagement.Specifically,transformer model is used to extract text semantic features,FasterRCNN model is used to extract visual features,and attention feature fusion is used to obtain the prediction classification results of different dimensions of learning engagement.Finally,visualization technology is used to present the picture of learning engagement.The research shows that the model effectively represents the unbalanced performance of learning groups in all dimensions,helps to carry out teaching diagnosis and collaborative grouping,effectively assists teachers in teaching diagnosis,and improves the teaching efficiency of collaborative grouping activities.