面向知识场景的图片类教育资源知识点自动标注算法

Automatic Annotation of Knowledge Points in Picture-Based Educational Resources for Knowledge Scenarios

针对图片资源的视觉特征与高级知识语义不一致的挑战,提出一种新的知识点自动标注算法,称为基于知识场景的情境超图卷积网络(SHGCN),以便高效组织管理教育领域中的图片数据,促进知识理解与有效利用,实现教育智能化。该算法在提取图片资源显性视觉特征的同时,又挖掘了隐含在细粒度区域的隐性知识信息。利用Faster R-CNN和OCR技术来识别知识对象和坐标文本等知识实体,这些知识实体特征融合后作为该图片的知识向量;提出双筛选机制来生成不同类型的知识场景,并将知识场景作为超边来构建情境超图,建模蕴含相似情境信息的图片间高阶知识相关性。利用超图卷积实现知识相似图片的情境信息聚合,实现“视觉-语义”到“视觉-语义-知识”的转化。还构建了一个物理学科的图片数据集来训练和验证SHGCN。实验结果表明,SHGCN在提取图片显性视觉信息的基础上,进一步挖掘隐性知识信息,其性能优于基线方法。

Aiming at the challenge of inconsistency between the visual features of picture resources and the semantics of advanced knowledge,a new automatic annotation algorithm for knowledge points is proposed,called the situational hyper-graph convolutional network based on knowledge scenarios(SHGCN),which can efficiently organize and manage picture data,promote knowledge understanding and utilization,and improve education intelligence.The algorithm not only extracts explicit visual features of the picture resources,but also mines knowledge information hidden in fine-grained regions.Faster R-CNN and OCR techniques are utilized to identify knowledge entities such as knowledge objects and coordinate texts,and multi-granularity features are fused to generate knowledge vectors.Then,a dual-screening mechanism is pro-posed to construct different types of knowledge scenarios,and the knowledge scenarios are used as hyperedges to con-struct a situational hypergraph to model higher-order knowledge correlations between images containing similar knowl-edge information.Finally,the hypergraph convolution is used to complete the information aggregation of knowledge-similar pictures,and realize the transformation from“vision-semantic”to“vision-semantic-knowledge”.This paper also con-structs a physical picture dataset to train and validate SHGCN.Experimental results show that SHGCN outperforms cur-rent state-of-the-art methods by fusing explicit visual features and implicit knowledge information of pictures.