多模态数据融合:破解智能教育关键问题的核心驱动力

Multimodal Data Fusion:The Core Driving Force to Solve the Key Problems of Intelligent Education

多模态数据融合旨在利用不同模态数据之间的信息互补机制提升数据分析的准确性,实现对学习主体和学习情境的精准刻画,进而还原教学过程全貌,挖掘深层次的教育规律,其已逐渐成为智能教育领域重要的技术方法和研究思想。智能教育领域常见的多模态数据类型包括外在行为表征数据、内在神经生理信息数据、人机交互数据以及学习情境感知数据。多模态数据的融合策略主要包括数据级融合、特征级融合和决策级融合,在数据分析的不同阶段选取恰当的融合策略,可以提升数据分析的准确性。在智能教育领域,多模态数据融合主要应用在人机交互分析、学习者情绪识别、学习投入分析、学业表现预测、学习情境感知五个方面。充分发挥多模态数据在学习过程感知和建模中的核心作用,可以实现对学习过程的有效还原和对学习规律的科学解释。多模态数据融合充分体现了基于数据密集型科学的教育科学研究范式变革,未来应着力于面向多元学习主体和学习情境的全时空多维度数据采集、基于多模态数据融合的学习者认知发展规律研究、基于多模态数据感知与融合的智能教育产品研发以及多模态数据采集的技术伦理问题等四个方面,构建智能时代教育科学研究的新样态。

Multimodal data fusion aims to improve the accuracy of data analysis by using the information complementation mechanism between different modal data,realize the accurate description of the learning subject and learning situation,and then restore the whole picture of the teaching process,and excavate the deep-level educational laws.It has gradually become an important technical method and research idea in the field of intelligent education.The common multimodal data types in the field of intelligent education include external behavior representation data,intrinsic neurophysiological information data,human-computer interaction data,and learning context awareness data.The fusion strategy of multimodal data consists of data-level fusion,feature-level fusion and decision-level fusion,and appropriate fusion strategies can be selected at different stages of data analysis to improve the accuracy of data analysis.In the field of intelligent education,multimodal data fusion is mainly used in five aspects:human-computer interaction analysis,learner emotion recognition,learning engagement analysis,academic performance prediction,and learning context awareness,which aims to give full play to the core role of multimodal data in the perception and modeling of the learning process,so as to achieve effective restoration of the learning process and scientific interpretation of learning laws.Multimodal data fusion fully embodies the paradigm change of scientific research on education based on data-intensive science.We should focus on the following four aspects in the future:full-time,multi-dimensional data collection for diverse learning subjects and learning situations,the research on the laws of learners’cognitive development based on multimodal data fusion,the development of intelligent education products based on multimodal data perception and fusion,and the technical ethics of multimodal data collection.In this way,a new state of scientific research on education in the intelligent era will be reconstructed.