智能教室中情境感知的多通道融合方法

Multichannel Fusion Model of Context Aware in Intelligent Classroom

现有的智能教室中多通道融合方法普遍缺乏情境信息的感知能力,融合策略固定、简单,不能很好解决多通道输入的二义性、非精确、冲突性和时间偏序关系。针对以上问题,采用EMMA标注语言调整时序关系,用层次任务网络规划器HTN规划动作行为,用证据理论融合各个情感检测通道的检测结果,提出了一种通用可扩展的基于情境感知的多通道融合模型及方法。实验结果表明,该方法较好地解决了多通道学生情感检测的冲突性、二义性,提高了检测的精确性与正确性。

Human computer interaction multichannel-model based on video, audio, keyboard is prevalent in the intelligent classrooms available for teaching, but the existed human-computer interaction multichannel models generally lack context awareness and self-adaption, and its fusion strategy is fixed and simple. Therefore, they can not solve the problems of ambiguity, dubiety, conflict and temporal relations of multichannel inputs. Focusing on those problems, a context aware multi-channel fusion model which is universal and extensible is proposed in this paper. In the model, EMMA （extensible multi modal annotation） markup language is adopted to adjust temporal relations of multichannel inputs, HTN （hierarchical task net- work） is used to plan the input behavior, and evidence theory is employed to fuse the detection results of multichannel. Th.e result of this experiment proves that this method can solve the dubiety and conflict in multichannel students＇ emotion detection, and improve the accuracy and correctness of the detection result.