A Method of Multimodal Emotion Recognition in Video Learning Based on Knowledge Enhancement

With the popularity of online learning and due to the significant influence of emotion on the learning effect,more and more researches focus on emotion recognition in online learning.Most of the current research uses the comments of the learning platform or the learner’s expression for emotion recognition.The research data on other modalities are scarce.Most of the studies also ignore the impact of instructional videos on learners and the guidance of knowledge on data.Because of the need for other modal research data,we construct a synchronous multimodal data set for analyzing learners’emotional states in online learning scenarios.The data set recorded the eye movement data and photoplethysmography(PPG)signals of 68 subjects and the instructional video they watched.For the problem of ignoring the instructional videos on learners and ignoring the knowledge,a multimodal emotion recognition method in video learning based on knowledge enhancement is proposed.This method uses the knowledge-based features extracted from instructional videos,such as brightness,hue,saturation,the videos’clickthrough rate,and emotion generation time,to guide the emotion recognition process of physiological signals.This method uses Convolutional Neural Networks(CNN)and Long Short-Term Memory(LSTM)networks to extract deeper emotional representation and spatiotemporal information from shallow features.The model uses multi-head attention(MHA)mechanism to obtain critical information in the extracted deep features.Then,Temporal Convolutional Network(TCN)is used to learn the information in the deep features and knowledge-based features.Knowledge-based features are used to supplement and enhance the deep features of physiological signals.Finally,the fully connected layer is used for emotion recognition,and the recognition accuracy reaches 97.51%.Compared with two recent researches,the accuracy improved by 8.57%and 2.11%,respectively.On the four public data sets,our proposed method also achieves better results compared with the two recent researches.The experiment results show that the proposed multimodal emotion recognition method based on knowledge enhancement has good performance and robustness.

Application of graph neural network and feature information enhancement in relation inference of sparse knowledge graph

At present,knowledge embedding methods are widely used in the field of knowledge graph(KG)reasoning,and have been successfully applied to those with large entities and relationships.However,in research and production environments,there are a large number of KGs with a small number of entities and relations,which are called sparse KGs.Limited by the performance of knowledge extraction methods or some other reasons(some common-sense information does not appear in the natural corpus),the relation between entities is often incomplete.To solve this problem,a method of the graph neural network and information enhancement is proposed.The improved method increases the mean reciprocal rank(MRR)and Hit@3 by 1.6%and 1.7%,respectively,when the sparsity of the FB15K-237 dataset is 10%.When the sparsity is 50%,the evaluation indexes MRR and Hit@10 are increased by 0.8%and 1.8%,respectively.

Traffic Scene Captioning with Multi-Stage Feature Enhancement

Traffic scene captioning technology automatically generates one or more sentences to describe the content of traffic scenes by analyzing the content of the input traffic scene images,ensuring road safety while providing an important decision-making function for sustainable transportation.In order to provide a comprehensive and reasonable description of complex traffic scenes,a traffic scene semantic captioningmodel withmulti-stage feature enhancement is proposed in this paper.In general,the model follows an encoder-decoder structure.First,multilevel granularity visual features are used for feature enhancement during the encoding process,which enables the model to learn more detailed content in the traffic scene image.Second,the scene knowledge graph is applied to the decoding process,and the semantic features provided by the scene knowledge graph are used to enhance the features learned by the decoder again,so that themodel can learn the attributes of objects in the traffic scene and the relationships between objects to generate more reasonable captions.This paper reports extensive experiments on the challenging MS-COCO dataset,evaluated by five standard automatic evaluation metrics,and the results show that the proposed model has improved significantly in all metrics compared with the state-of-the-art methods,especially achieving a score of 129.0 on the CIDEr-D evaluation metric,which also indicates that the proposed model can effectively provide a more reasonable and comprehensive description of the traffic scene.

SHEL:a semantically enhanced hardware-friendly entity linking method

With the help of pre-trained language models,the accuracy of the entity linking task has made great strides in recent years.However,most models with excellent performance require fine-tuning on a large amount of training data using large pre-trained language models,which is a hardware threshold to accomplish this task.Some researchers have achieved competitive results with less training data through ingenious methods,such as utilizing information provided by the named entity recognition model.This paper presents a novel semantic-enhancement-based entity linking approach,named semantically enhanced hardware-friendly entity linking(SHEL),which is designed to be hardware friendly and efficient while maintaining good performance.Specifically,SHEL's semantic enhancement approach consists of three aspects:(1)semantic compression of entity descriptions using a text summarization model;(2)maximizing the capture of mention contexts using asymmetric heuristics;(3)calculating a fixed size mention representation through pooling operations.These series of semantic enhancement methods effectively improve the model's ability to capture semantic information while taking into account the hardware constraints,and significantly improve the model's convergence speed by more than 50%compared with the strong baseline model proposed in this paper.In terms of performance,SHEL is comparable to the previous method,with superior performance on six well-established datasets,even though SHEL is trained using a smaller pre-trained language model as the encoder.

Knowledge Graph Enhanced Transformers for Diagnosis Generation of Chinese Medicine

Chinese medicine(CM)diagnosis intellectualization is one of the hotspots in the research of CM modernization.The traditional CM intelligent diagnosis models transform the CM diagnosis issues into classification issues,however,it is difficult to solve the problems such as excessive or similar categories.With the development of natural language processing techniques,text generation technique has become increasingly mature.In this study,we aimed to establish the CM diagnosis generation model by transforming the CM diagnosis issues into text generation issues.The semantic context characteristic learning capacity was enhanced referring to Bidirectional Long Short-Term Memory(BILSTM)with Transformer as the backbone network.Meanwhile,the CM diagnosis generation model Knowledge Graph Enhanced Transformer(KGET)was established by introducing the knowledge in medical field to enhance the inferential capability.The KGET model was established based on 566 CM case texts,and was compared with the classic text generation models including Long Short-Term Memory sequence-to-sequence(LSTM-seq2seq),Bidirectional and Auto-Regression Transformer(BART),and Chinese Pre-trained Unbalanced Transformer(CPT),so as to analyze the model manifestations.Finally,the ablation experiments were performed to explore the influence of the optimized part on the KGET model.The results of Bilingual Evaluation Understudy(BLEU),Recall-Oriented Understudy for Gisting Evaluation 1(ROUGE1),ROUGE2 and Edit distance of KGET model were 45.85,73.93,54.59 and 7.12,respectively in this study.Compared with LSTM-seq2seq,BART and CPT models,the KGET model was higher in BLEU,ROUGE1 and ROUGE2 by 6.00–17.09,1.65–9.39 and 0.51–17.62,respectively,and lower in Edit distance by 0.47–3.21.The ablation experiment results revealed that introduction of BILSTM model and prior knowledge could significantly increase the model performance.Additionally,the manual assessment indicated that the CM diagnosis results of the KGET model used in this study were highly consistent with the practical diagnosis results.In conclusion,text generation technology can be effectively applied to CM diagnostic modeling.It can effectively avoid the problem of poor diagnostic performance caused by excessive and similar categories in traditional CM diagnostic classification models.CM diagnostic text generation technology has broad application prospects in the future.

Knowledge-Enhanced Conversational Agents

Humanity has fantasized about artificial intelligence tools able to discuss with human beings fluently for decades.Numerous efforts have been proposed ranging from ELIZA to the modern vocal assistants.Despite the large interest in this research and innovation field,there is a lack of common understanding on the concept of conversational agents and general over expectations that hide the current limitations of existing solutions.This work proposes a literature review on the subject with a focus on the most promising type of conversational agents that are powered on top of knowledge bases and that can offer the ground knowledge to hold conversation autonomously on different topics.We describe a conceptual architecture to define the knowledge-enhanced conversational agents and investigate different domains of applications.We conclude this work by listing some promising research pathways for future work.

Enhancing Students’ Learning about Healthy Living through Community Participation

Introduction: The Health Enhancement Module (HEM) is taught as a core curriculum for all medical students at Monash University since 2002. In 2012 we moved the year three content of the program into a community setting, calling it the Health Enhancement Carnival (HEC). At the carnival, our undergraduates interacted with school students, their teachers, and their parents, involving them in a mix of discussions, poster presentations, and video presentations. In this paper we present our experience with the HEC. Specifically, we looked at the following two measures: how did the HEC influence the knowledge, attitude, and practice of healthy living among medical students? And, what were the learning experiences of the students during the HEC? Methods: Five themes (exercise, food, healthy sleep, workplace stress and ageing) were divided among students. They were asked to develop those themes with the help of posters, power point presentations, community talks as well as video presentations. The carnival was held in the setting of two nearby children’s schools. Students were evaluated by a panel of examiners with regards to learning objectives as well as preparation and presentation. As part of evaluation, we developed 2 questionnaires. The HEP Healthy Living Questionnaire provided feedback on how the program had improved students’ knowledge, attitudes, and practice of healthy living. The HEP Learning Style Questionnaire covered twelve areas, including collegiality, environment, leadership, community interaction and other facets of learning style. Analyses were performed using the IBM SPSS Statistics version 20 software in the Clinical School Johor Bahru. Results: 1) Influence of HEC on the knowledge, attitude, and practice of healthy living among medical students. From the interviews, the judges gave the students mean ratings of 4.0/5. We also received 77 out of 127 feedback questionnaires (response rate: 60.6%) from the students. Most students (range: 49.35% to 55.84%) were “satisfied/totally satisfied”, “achieved/totally achieved”, or “improved/totally improved” to 5 questions of the Healthy Living Questionnaire. Correlation coefficients between knowledge of healthy living, attitude towards healthy living, and practice of healthy living were large (exceeding 0.8) suggesting that these three measures were highly and positively inter-correlated. Most students (range: 60.28% to 71.43%) scored “a lot/almost all”, to 5 questions regarding achievement of learning objectives. 2) Learning experiences of the students during the HEC. Responding to the HEP Learning Style Questionnaire, most students (range: 66.24% to 85.72%) agreed or strongly agreed that the program provided an optimal environment for learning, encouraging students to assume leadership responsibilities and promoting self-directed learning. A correlation matrix of the 12 items showed medium to large correlations between all twelve variables. Conclusions: The Health Enhancement Program (HEP) is an innovative approach that has enabled students to learn about healthy living within the context of the local community.

Improving Extraction of Chinese Open Relations Using Pre-trained Language Model and Knowledge Enhancement

Open Relation Extraction(ORE)is a task of extracting semantic relations from a text document.Current ORE systems have significantly improved their efficiency in obtaining Chinese relations,when compared with conventional systems which heavily depend on feature engineering or syntactic parsing.However,the ORE systems do not use robust neural networks such as pre-trained language models to take advantage of large-scale unstructured data effectively.In respons to this issue,a new system entitled Chinese Open Relation Extraction with Knowledge Enhancement(CORE-KE)is presented in this paper.The CORE-KE system employs a pre-trained language model(with the support of a Bidirectional Long Short-Term Memory(BiLSTM)layer and a Masked Conditional Random Field(Masked CRF)layer)on unstructured data in order to improve Chinese open relation extraction.Entity descriptions in Wikidata and additional knowledge(in terms of triple facts)extracted from Chinese ORE datasets are used to fine-tune the pre-trained language model.In addition,syntactic features are further adopted in the training stage of the CORE-KE system for knowledge enhancement.Experimental results of the CORE-KE system on two large-scale datasets of open Chinese entities and relations demonstrate that the CORE-KE system is superior to other ORE systems.The F1-scores of the CORE-KE system on the two datasets have given a relative improvement of 20.1%and 1.3%,when compared with benchmark ORE systems,respectively.The source code is available at https:/github.COm/cjwen15/CORE-KE.

基于多模板提示调优和知识增强的事件因果关系识别方法

事件因果关系识别(Event Causality Identification,ECI)是自然语言处理领域的一项重要研究任务,旨在识别文本中事件之间的因果关系。现有方法大都基于微调范式,不能较好发挥预训练语言模型的作用,难以有效捕获隐式因果关系识别的线索。为此,该文提出了一种基于多模板提示调优和知识增强的事件因果关系识别方法。针对ECI任务设计独特的总提示模板,对显式和隐式事件因果关系分别设计不同的种子提示模板,集成训练所有提示模板,形成适应于ECI任务的提示调优方式。通过引入ConceptNet、Oxford Dictionaries等外部知识库,丰富事件的解释性知识和事件之间的关系性知识,将不同的知识融入提示模板,强化隐式因果关系线索。在EventStoryLine和Causal-TimeBank两个广泛使用的数据集上的实验结果表明,该文方法性能优于现有方法。

加速康复外科领域国内外研究现状及趋势对比分析

目的:基于文献计量学探讨国内外加速康复外科(enhanced recovery after surgery,ERAS)领域的研究热点及发展趋势。方法:利用中国国家知识基础设施(China National Knowledge Infrastructure,CNKI)检索平台、万方数据知识服务平台和Web of Science核心合集检索相关中外文文献,检索时间范围为建库至2023年2月10日,使用Citespace 6.1.R6、VOSviewer 1.6.13软件可视化分析年发文量、期刊、作者、国家、机构、学科和关键词信息。结果:共纳入中文6166篇、外文6212篇。ERAS相关中外文献的年发文量均呈逐年上升趋势;《齐鲁护理杂志》和Surgical Endoscopy and Other Interventional Techniques分别为国内外发文量最多的期刊,前10位共被引期刊的2022年期刊影响因子(Journal Impact Factor,JIF)多高于5分;美国、中国、英国的外文发文量排名前三,且差距明显,美国对外合作最为广泛;四川大学华西医院和哥本哈根大学发文量最多,伦敦大学学院是对外合作最密切的科研机构;国内外发文量最多的作者分别为江志伟和Kehlet H;学科主要分布在外科学、麻醉学、胃肠病学与肝病学等;中外文高频关键词为围手术期、腹腔镜、结直肠肿瘤、胃肠肿瘤、并发症、length of stay、colorectal surgery、perioperative care和postoperative complications等;研究热点集中于ERAS的外科应用、围手术期管理、应用效果和循证研究;达芬奇机器人、并发症、生活质量、预康复、循证护理、multimodal analgesia、pain management、same-day discharge等突现词体现了研究前沿。结论:国内ERAS的研究内容以临床/护理应用效果为主,同质性较高,创新性略显不足,缺乏大样本前瞻性研究,且实施过程存在困难和瓶颈,需重视高质量研究的开展,借鉴国外ERAS专业团队的成功经验,探索具有中国特色的ERAS发展之路。

妇科护士加速康复外科理念知信行现状的调查研究

目的了解妇科护士加速康复外科(enhanced recovery after surgery,ERAS)理念知信行现状。方法2023年8月,采取便利抽样法和滚雪球抽样法,选取来自20个省份的1553名妇科护士作为调查对象,采用一般资料调查表、妇科护士ERAS理念知信行问卷对其进行调查。结果共回收有效问卷1470份,有效问卷回收率为94.66%。1470名妇科护士ERAS理念知信行得分分别为(15.75±3.53)分、(23.57±2.62)分、(94.58±19.39)分。所在医院类别、所在科室床位数、ERAS实施时间、ERAS相关指南内容知晓情况、所在科室ERAS相关规章或流程拥有情况、成立ERAS小组情况、组织开展ERAS相关培训情况不同的妇科护士ERAS理念知信行得分比较,差异均有统计学意义(P<0.05)。结论妇科护士对ERAS的态度较为积极,但知识、行为水平仍有较大提升空间。护理管理者应注重对妇科护士ERAS理念知信行水平的评估,并基于妇科护士ERAS理念知信行水平特征对其进行针对性的干预,进而提高其ERAS理念知信行水平,保障ERAS在妇科手术围术期管理中的顺利实施,提升其实施效果。

融合外部知识图谱的多模态知识共创价值识别研究

[目的/意义]快速且精准地识别海量多模态数据中的价值性内容,对于促进知识传播、提升产出质量具有重要的意义。[方法/过程]基于可解释性视角聚焦知识共创中“用户+多模态知识”的双重推进机制。首先,依托BERT+BiLSTM与ResNet模型分别提炼文本与图片特征以获取多模态知识向量表示;其次,依据社会认知理论剖析用户行为,采用DeepFM捕捉交互特征间的关联生成用户向量表示;再次,借助K-BERT对文本数据嵌入知识图谱得到外部知识向量表示;最后,基于多头注意力机制融合各维度特征向量,通过动态调整权重完成价值内容的识别。[结果/结论]通过使用魅族Flyme社区数据进行实验,所构建的融合模型准确率达到88.31%,相较于其他基线模型与组合模型,评价指标均有不同程度的提升,证明嵌入外部知识并融合文本、图片与用户属性可以有效提升价值的识别效果。

Seru系统调度优化的知识引导协同进化算法

作为一种新型的生产模式, Seru系统能够兼顾柔性和效率且快速响应市场,已在装配企业得到广泛应用.为了实现实际生产过程生产效率和劳动效率的协同优化,本文研究以最小化最大完工时间和工人总劳动时间为目标的Seru系统多目标调度问题,提出一种知识引导的协同进化算法.首先,将问题分解为Seru构造和Seru调度,构造两个种群分别优化子问题.同时,设计种群规模的调整策略,通过为有潜力的种群分配更多个体来提高协同搜索的效率.进而,通过分析问题的性质,提炼规则性知识用于设计有效的搜索算子和重生成规则,指导精英个体执行知识驱动的增强搜索,从而进一步提升算法的局部开发能力.通过数值仿真和统计性能对比,验证了算法各设计环节的有效性,并取得了显著优于现有最新算法的多目标调度优化性能.

融合SVM-LDA与加权相似度的潜在新兴技术识别研究——以人工智能领域为例

在新一轮科技革命和产业变革加速发展的大背景下,如何在新技术不断涌现的技术大海中精准找到和识别出有颠覆性潜力的新兴技术,对于国家、企业参与主体和相关商业投资机构把握科技创新发展趋势和方向、合理配置科技资源、提前进行科技战略规划与技术布局具有重要的意义。本文提出一种基于知识增强SVM-LDA(Support Vector Machine-Latent Dirichlet Allocation)的新兴技术主题识别模型。首先,基于专家小组的先验知识,制定基础技术类别划分标准;其次,将技术类别划分标准作为先验知识输入SVM-LDA模型,得到技术主题聚类结果;再其次,基于类别主题词的加权相似度计算,确定潜在新兴关键技术;最后,以人工智能领域为例进行实证研究。采用本文模型共得到24项潜在新兴技术,主要分布在特种机器人技术、监测预警技术、视频图像处理技术、语音识别技术、自动规划和决策技术以及自然语言处理技术6个大类方向。

新质生产力需求视域下高校教师能力提升策略

构建高水平教师队伍是培养创新型人才、夯实新质生产力发展的人才基础、推进教育强国和科技强国建设的前提和保障。新质生产力发展对高校教师的专业知识更新能力、跨学科融合能力、数字化教学能力、创新实践及成果转化能力、国际学术交流与合作能力等方面提出更高要求。然而,我国高校教师普遍存在诸多能力上的不足,在一定程度上制约了新质生产力的发展。通过建立完善的教师培训体系、搭建优质的跨界合作平台、深入推进国际合作与交流、优化激励机制与评价体系等措施,可以有效提高教师的整体素质和能力水平,进而更好地服务于新质生产力的发展需求。

慢性鼻-鼻窦炎内镜手术患者加速康复外科护理知信行现状调查

目的调查慢性鼻-鼻窦炎患者对鼻内镜手术加速康复外科(ERAS)护理的知识、态度及行为依从状况,并分析影响因素。方法选取2023年2月至2023年6月重庆市某三级甲等综合医院耳鼻咽喉头颈外科病房147例行内镜鼻窦手术治疗的慢性鼻-鼻窦炎患者作为研究对象,采用自制调查问卷调查患者一般情况与围手术期ERAS护理有关知识、态度及行为情况。结果共回收问卷156份,有效问卷147份,有效回收率为94.23%。147例患者中,鼻内镜手术ERAS护理知识得分率为46.36%、态度75.60%、行为71.25%,知信行总体得分率为62.11%,处于中等偏下水平。多因素分析结果显示,疾病病程和既往鼻部手术史是慢性鼻-鼻窦炎内镜手术患者ERAS护理知识水平的主要影响因素,性别、疾病病程是鼻内镜手术ERAS护理态度的主要影响因素,性别是鼻内镜手术ERAS护理依从性的主要影响因素。结论慢性鼻-鼻窦炎内镜手术患者对ERAS护理理念的认知及行为依从性较差,临床上应加强患者的宣传教育,提高手术疗效。

矿山行业大模型建设路径探索与应用展望

煤炭是保障能源安全的压舱石。在当前加快发展数字经济、积极稳妥推进“双碳”目标的背景下,煤炭行业亟需深化数字化转型与智能化建设。在此背景下,探索引入大模型技术赋能煤炭行业应用,充分利用行业海量知识数据,加快推动煤炭行业的数字化发展,已成为行业关注的焦点。基于此,梳理了通用大模型技术的发展现状,阐述了大模型技术在多领域的应用现状与成效,介绍了数据处理(清洗、平衡、增强等)、文本分词、预训练与微调、提示词优化、向量嵌入、对齐、检索增强生成等行业大模型关键技术,表明了行业大模型在继承通用大模型“通”的优势的同时又兼具“专”的特点,在推动行业生产力革新和产业升级方面发挥着重要作用。深度剖析了大模型技术在煤炭行业应用面临研发投入成本高、高质量数据搜集难度大、多模态数据融合技术难度高等挑战,从基础设施层、数据资源层、算法模型层、应用服务层、安全可信与测试层、行业生态层六方面详细总结了太阳石矿山大模型为应对上述挑战采取的建设路径以及取得的阶段性成效,最后对大模型技术的发展给煤炭行业带来的生产与技术变革进行了展望,指出矿山行业大模型建设应遵循开源模型与行业数据相结合的路径,发挥大模型的工具属性以赋能业务场景、构建“产-学-研-用”相结合的应用生态,助力矿山行业新质生产力的发展。

基于文本知识增强的问题生成模型

预训练语言模型在大规模训练数据和超大规模算力的基础上,能够从非结构化的文本数据中学到大量的知识。针对三元组包含信息有限的问题,提出利用预训练语言模型丰富知识的问题生成方法。首先,利用预训练语言模型中丰富的知识增强三元组信息,设计文本知识生成器,将三元组中的信息转化为子图描述,丰富三元组的语义;然后,使用问题类型预测器预测疑问词,准确定位答案所在的领域,从而生成语义正确的问题,更好地控制问题生成的效果;最后,设计一种受控生成框架对关键实体和疑问词进行约束,保证关键实体和疑问词同时出现在问题中,使生成的问题更加准确。在公开数据集WebQuestion和PathQuestion中验证所提模型的性能。实验结果表明,与现有模型LFKQG相比,所提模型的BLUE-4、METEOR、ROUGE-L指标在WebQuestion数据集上分别提升0.28、0.16、0.22个百分点,在PathQuestion数据集上分别提升0.8、0.39、0.46个百分点。

贵阳市某三甲医院妇产科医护人员加速康复外科知信行现状及讲座的干预效果

目的 了解贵阳市某三甲医院妇产科医护人员加速康复外科(ERAS)知信行现状,探讨讲座在改善妇产科医护人员加速康复知信行中的效果。方法 以贵阳市某三甲医院妇产科146名医护人员为研究对象,自行设计调查问卷,计算问卷各维度得分评价调查对象ERAS知信行状态,采用单因素方差分析妇产科医护人员ERAS知信行得分;对妇产科医护人员进行1次讲座形式的知识宣教,比较讲座前后妇产科医护人员ERAS知信行得分变化情况。结果 讲座前妇产科医护人员加速康复外科知识得分较低,得分率仅为65.74%;单因素方差分析和独立样本t检验显示,研究对象知信行得分在学历、年龄、科室、职称、工作年限方面比较,差异有统计意义(P<0.05);讲座后妇产科医护人员加速康复外科知信行整体得分及知识、态度、行为各项单独得分均较讲座前提高(P<0.05)。结论 妇产科医护人员加速康复外科知识得分较低,对妇产科医护人员进行讲座形式的培训后,可有效提高医护人员的知信行状况。

基于知识增强的文本语义匹配模型研究

文本语义匹配模型在信息检索、文本挖掘等领域已经获得了广泛应用。为解决现有模型主要从文本自身角度判断文本之间的语义关系而忽略对外部知识有效利用的问题,本文提出一种新的基于知识增强的文本语义匹配模型,以知识图谱实体作为外部知识,有效建模文本的外部知识信息,并自适应地过滤外部知识中存在的噪声。针对自然语言推理和释义识别两个文本语义匹配任务,与基线方法相比,本文模型在大多数指标上取得了最优效果。研究结果表明,本文模型有助于揭示知识图谱在文本语义匹配任务中的作用,为将知识图谱应用到智能信息服务领域提供了参考。