PAL-BERT:An Improved Question Answering Model

In the field of natural language processing(NLP),there have been various pre-training language models in recent years,with question answering systems gaining significant attention.However,as algorithms,data,and computing power advance,the issue of increasingly larger models and a growing number of parameters has surfaced.Consequently,model training has become more costly and less efficient.To enhance the efficiency and accuracy of the training process while reducing themodel volume,this paper proposes a first-order pruningmodel PAL-BERT based on the ALBERT model according to the characteristics of question-answering(QA)system and language model.Firstly,a first-order network pruning method based on the ALBERT model is designed,and the PAL-BERT model is formed.Then,the parameter optimization strategy of the PAL-BERT model is formulated,and the Mish function was used as an activation function instead of ReLU to improve the performance.Finally,after comparison experiments with traditional deep learning models TextCNN and BiLSTM,it is confirmed that PALBERT is a pruning model compression method that can significantly reduce training time and optimize training efficiency.Compared with traditional models,PAL-BERT significantly improves the NLP task’s performance.

Operational requirements analysis method based on question answering of WEKG

The weapon and equipment operational requirement analysis(WEORA) is a necessary condition to win a future war,among which the acquisition of knowledge about weapons and equipment is a great challenge. The main challenge is that the existing weapons and equipment data fails to carry out structured knowledge representation, and knowledge navigation based on natural language cannot efficiently support the WEORA. To solve above problem, this research proposes a method based on question answering(QA) of weapons and equipment knowledge graph(WEKG) to construct and navigate the knowledge related to weapons and equipment in the WEORA. This method firstly constructs the WEKG, and builds a neutral network-based QA system over the WEKG by means of semantic parsing for knowledge navigation. Finally, the method is evaluated and a chatbot on the QA system is developed for the WEORA. Our proposed method has good performance in the accuracy and efficiency of searching target knowledge, and can well assist the WEORA.

A Study of the Treatment of Disfiguring Diseases in the “Qian Jin Fang”

Traditional Chinese medicine(TCM)has been passed down for more than 2,000 years.TCM beauty products driven by the principles of identification and treatment with various dosage forms and exhibit mild and safe efficacy are bound to become a main component of the beauty industry in the future.Xu once commented that“Qian Jin Fang”is different from the traditional prescription system,which emphasizes the use of medicines.In this paper,we mainly selected the prescriptions for the treatment of disfigurement diseases such as acne,scarring,black dryness,face hyperpigmentation,black mole,and rosacea from the article titled,“Upper Seven Orifices Disease-Facial Medicine IX”in the Bei Ji Qian Jin Yao Fang[1].By studying and analyzing the original composition of the formula,understanding the etiology and pathogenesis of the disease,and exploring the characteristics of the formula concerning the efficacy of TCM,their properties,flavors,and meridians,we summarized the characteristics of the Qian Jin Fang.It is based on the external treatment of disfigurement diseases,with a wide variety of drugs,flexibility,and accuracy.This method draws upon analogies while utilizing excipients and harmonizers to focus on the combined healing approach of treatment and nourishment.

Analysis of community question-answering issues via machine learning and deep learning:State-of-the-art review

Over the last couple of decades,community question-answering sites(CQAs)have been a topic of much academic interest.Scholars have often leveraged traditional machine learning(ML)and deep learning(DL)to explore the ever-growing volume of content that CQAs engender.To clarify the current state of the CQA literature that has used ML and DL,this paper reports a systematic literature review.The goal is to summarise and synthesise the major themes of CQA research related to(i)questions,(ii)answers and(iii)users.The final review included 133 articles.Dominant research themes include question quality,answer quality,and expert identification.In terms of dataset,some of the most widely studied platforms include Yahoo!Answers,Stack Exchange and Stack Overflow.The scope of most articles was confined to just one platform with few cross-platform investigations.Articles with ML outnumber those with DL.Nonetheless,the use of DL in CQA research is on an upward trajectory.A number of research directions are proposed.

ALBERT with Knowledge Graph Encoder Utilizing Semantic Similarity for Commonsense Question Answering

Recently,pre-trained language representation models such as bidirec-tional encoder representations from transformers(BERT)have been performing well in commonsense question answering(CSQA).However,there is a problem that the models do not directly use explicit information of knowledge sources existing outside.To augment this,additional methods such as knowledge-aware graph network(KagNet)and multi-hop graph relation network(MHGRN)have been proposed.In this study,we propose to use the latest pre-trained language model a lite bidirectional encoder representations from transformers(ALBERT)with knowledge graph information extraction technique.We also propose to applying the novel method,schema graph expansion to recent language models.Then,we analyze the effect of applying knowledge graph-based knowledge extraction techniques to recent pre-trained language models and confirm that schema graph expansion is effective in some extent.Furthermore,we show that our proposed model can achieve better performance than existing KagNet and MHGRN models in CommonsenseQA dataset.

Structure of the Quarks and a New Model of Protons and Neutrons: Answer to Some Open Questions

The described structural model tries to answer some open questions such as: Why do quarks not exist in the open state? Where are the antiparticles from the Big Bang?

Information Extraction Based on Multi-turn Question Answering for Analyzing Korean Research Trends

Analyzing Research and Development(R&D)trends is important because it can influence future decisions regarding R&D direction.In typical trend analysis,topic or technology taxonomies are employed to compute the popularities of the topics or codes over time.Although it is simple and effective,the taxonomies are difficult to manage because new technologies are introduced rapidly.Therefore,recent studies exploit deep learning to extract pre-defined targets such as problems and solutions.Based on the recent advances in question answering(QA)using deep learning,we adopt a multi-turn QA model to extract problems and solutions from Korean R&D reports.With the previous research,we use the reports directly and analyze the difficulties in handling them using QA style on Information Extraction(IE)for sentence-level benchmark dataset.After investigating the characteristics of Korean R&D,we propose a model to deal with multiple and repeated appearances of targets in the reports.Accordingly,we propose a model that includes an algorithm with two novel modules and a prompt.A newly proposed methodology focuses on reformulating a question without a static template or pre-defined knowledge.We show the effectiveness of the proposed model using a Korean R&D report dataset that we constructed and presented an in-depth analysis of the benefits of the multi-turn QA model.

Expert Recommendation in Community Question Answering via Heterogeneous Content Network Embedding

ExpertRecommendation(ER)aims to identify domain experts with high expertise and willingness to provide answers to questions in Community Question Answering(CQA)web services.How to model questions and users in the heterogeneous content network is critical to this task.Most traditional methods focus on modeling questions and users based on the textual content left in the community while ignoring the structural properties of heterogeneous CQA networks and always suffering from textual data sparsity issues.Recent approaches take advantage of structural proximities between nodes and attempt to fuse the textual content of nodes for modeling.However,they often fail to distinguish the nodes’personalized preferences and only consider the textual content of a part of the nodes in network embedding learning,while ignoring the semantic relevance of nodes.In this paper,we propose a novel framework that jointly considers the structural proximity relations and textual semantic relevance to model users and questions more comprehensively.Specifically,we learn topology-based embeddings through a hierarchical attentive network learning strategy,in which the proximity information and the personalized preference of nodes are encoded and preserved.Meanwhile,we utilize the node’s textual content and the text correlation between adjacent nodes to build the content-based embedding through a meta-context-aware skip-gram model.In addition,the user’s relative answer quality is incorporated to promote the ranking performance.Experimental results show that our proposed framework consistently and significantly outperforms the state-of-the-art baselines on three real-world datasets by taking the deep semantic understanding and structural feature learning together.The performance of the proposed work is analyzed in terms of MRR,P@K,and MAP and is proven to be more advanced than the existing methodologies.

Improved Blending Attention Mechanism in Visual Question Answering

Visual question answering(VQA)has attracted more and more attention in computer vision and natural language processing.Scholars are committed to studying how to better integrate image features and text features to achieve better results in VQA tasks.Analysis of all features may cause information redundancy and heavy computational burden.Attention mechanism is a wise way to solve this problem.However,using single attention mechanism may cause incomplete concern of features.This paper improves the attention mechanism method and proposes a hybrid attention mechanism that combines the spatial attention mechanism method and the channel attention mechanism method.In the case that the attention mechanism will cause the loss of the original features,a small portion of image features were added as compensation.For the attention mechanism of text features,a selfattention mechanism was introduced,and the internal structural features of sentences were strengthened to improve the overall model.The results show that attention mechanism and feature compensation add 6.1%accuracy to multimodal low-rank bilinear pooling network.

Deep Multi-Module Based Language Priors Mitigation Model for Visual Question Answering

The original intention of visual question answering(VQA)models is to infer the answer based on the relevant information of the question text in the visual image,but many VQA models often yield answers that are biased by some prior knowledge,especially the language priors.This paper proposes a mitigation model called language priors mitigation-VQA(LPM-VQA)for the language priors problem in VQA model,which divides language priors into positive and negative language priors.Different network branches are used to capture and process the different priors to achieve the purpose of mitigating language priors.A dynamically-changing language prior feedback objective function is designed with the intermediate results of some modules in the VQA model.The weight of the loss value for each answer is dynamically set according to the strength of its language priors to balance its proportion in the total VQA loss to further mitigate the language priors.This model does not depend on the baseline VQA architectures and can be configured like a plug-in to improve the performance of the model over most existing VQA models.The experimental results show that the proposed model is general and effective,achieving state-of-the-art accuracy in the VQA-CP v2 dataset.

The Taiwan Question China From the Perspective of International Crisis Management:History and Future

The Taiwan Question China discussed in this paper belongs to the theoretical crisis discussion on international relations and does not regard the Cross-Strait relations as relations between different countries.The outcome of the 2024 Taiwan Election has a great impact on the Taiwan question,the latest poll shows that the possibility of the Democratic Progressive Party(DPP)candidate to come to power is still very high,because its political evolution trend of Taiwan independence still exists.

对基础教育阶段未来拔尖创新人才培养的一些认识

培养未来拔尖创新人才,须从基础教育阶段培养孩子的创造力抓起。为探究基础教育阶段培养学生创新能力的方法,文章对“钱学森之问”和国外开展的创造力培养研究进行了梳理,发现国内外在基础教育阶段推进学生创造力培养的核心要素可归纳为七个方面:对基础教育阶段学生的创造力培养给予高度重视;让学生学会发现问题;培养创新型教师;改革课堂教学;训练思维方法;改革学生评价方式;强调培养拔尖创新人才。由此,建议以跨文化比较的视角,深入了解国内外培养儿童青少年创造力的实践经验,为提升我国学生创造力、促进基础教育阶段未来拔尖创新人才培养提供可靠依据。

基于粗糙分析的大学英语考试质量提升路径

大学英语考试是检验大学英语教学质量和效果的有效手段。将粗糙集理论应用于大学英语考试命题,基于粗糙集的相对正域、冗余属性和属性重要度等概念,给出试题冗余性的定性判别方法,提出试题重要度的定量度量方法,建立基于粗糙分析的大学英语考试质量提升模型。实验结果表明,新型方法发现并修正了部分冗余和重要度较低的试题,有效提高了试卷命题质量,对指导大学英语教学工作、提升教学质量具有重要指导意义。

旅游自动问答系统中多任务问句分类研究

目前旅游产业信息化建设需要构建旅游自动问答系统,其中问句分类是问答系统的重要组成部分,传统问句类别体系角度单一,且传统分类模型对不平衡的问句数据集表现欠佳。针对这一问题,该文从问题主题和问句答案类型两个角度构建了旅游领域的问句类别体系架构,并提出多任务问句分类模型MT-Bert,在BERT上进行多任务训练,并加入自注意力机制,使用Softmax分类器,并设计了多任务融合损失函数。在山西旅游数据集的结果表明,MT-Bert在两种类别体系的微平均F1值分别为97.6%、91.7%,且避免了非平衡数据的预测失败问题,可以有效处理非平衡数据。

一种面向中文自动问答的注意力交互深度学习模型

随着互联网、大数据的飞速发展,以深度神经网络(DNN)为代表的人工智能技术迎来了黄金发展时期,自动问答作为人工智能领域的一个重要分支,也得到越来越多学者的关注。现有网络模型可以提取问题或答案的语义特征,但其一方面忽略了问题与答案之间的语义联系,另一方面也不能从整体上把握问题或答案内部所有字符之间的潜在联系。基于此,提出了两种不同形式的注意力交互模块,即互注意力交互模块和自注意力交互模块,并设计出一套基于所提注意力交互模块的深度学习模型,用于证明该注意力交互模块的有效性。首先将问题和答案中的每个字符映射成固定长度的向量,分别得到问题和答案对应的字嵌入矩阵;然后将字嵌入矩阵送入注意力交互模块,得到综合考虑问题与答案所有字符之后的字嵌入矩阵,并与之前的字嵌入矩阵相加,送入深度神经网络模块,用于提取问题与答案的语义特征;最后得到问题与答案的向量表示并计算两者之间的相似度。实验结果表明,所提模型的Top-1准确度较主流深度学习模型最高提升了3.55%,证明了所提注意力交互模块对于改善上述问题的有效性。

On Answering the Question Related to the Legal Regulation of Working Hours in the New Era:Western Experience and Chinese Approach

The popularity of flexible working hours around the world has slowed down the historical trend of reducing working hours.It even shows signs of regression.Whether and how to guide the cur-rent society with flexible working hours to return to the historical track of reducing working hours,improve the quality of working hours,and promote a smooth transition from the era of traditional standard work-ing hours to the era of flexible working hours has become a question related to the legal regulation of working hours in the new era.In this regard,although Western countries have proposed new regulatory concepts and carried out legislative practices with distinctive charac-teristics,the limitations of legal regulation capabilities have prevented them from proposing a package of institutional solutions.The advan-tage of China in the ability of legal regulation of working hours has been gradually formed in the legislation on working hours unnder the leadership of the CPC in the past century.It enables China to break through the limitations of the West and propose a Chinese approach to answer the question of the legal regulation of working hours in the new era from three aspects:limiting the extension of working hours,improving the quality of flexible working hours,and optimizing the funnctions of the multi-funnctional regulatory system for working hours.

融入三维语义特征的常识推理问答方法

现有使用预训练语言模型和知识图谱的常识问答方法主要集中于构建知识图谱子图及跨模态信息结合的研究,忽略了知识图谱自身丰富的语义特征,且缺少对不同问答任务的知识图谱子图节点相关性的动态调整,导致预测准确率低。为解决以上问题,提出一种融入三维语义特征的常识推理问答方法。首先提出知识图谱节点的关系层级、实体层级、三元组层级三维语义特征量化指标;其次,通过注意力机制动态计算关系层级、实体层级、三元组层级三种维度的语义特征对不同实体节点间的重要性;最后,通过图神经网络进行多层聚合迭代嵌入三维语义特征,获得更多的外推知识表示,更新知识图谱子图节点表示,提升答案预测精度。与QA-GNN常识问答推理方法相比,所提方法在CommonsenseQA数据集上的验证集和测试集的准确率分别提高了1.70个百分点和0.74个百分点,在OpenBookQA数据集上使用AristoRoBERTa数据处理方法的准确率提高了1.13个百分点。实验结果表明,所提出的融入三维语义特征的常识推理问答方法能够有效提高常识问答任务准确率。

犹太人问题与自由民主政体——基于斯宾诺莎《神学政治论》的考察

犹太人问题由来已久,但第一次提出该问题的当属斯宾诺莎。在斯宾诺莎的时代,尼德兰沦为帝国主义势力的角逐场。结合这一时代问题与历史问题,斯宾诺莎在《神学政治论》中对于好的政体,即自由民主政体有着细致的描述与刻画。透过自由民主政体的表象深入斯宾诺莎建构理论的逻辑基地显得尤为重要。首先,斯宾诺莎在《神学政治论》中运用历史主义和文献研读的方法,重构诸多概念,让神学与哲学、教法与世俗法分道扬镳,粉碎了犹太教固有的“永久选民”的优越感。其次,在《神学政治论》中,斯宾诺莎从人性观与自由观两个角度对犹太民族的民族基础予以重击。最后,斯宾诺莎在重构神学、重解犹太民族的基础上,建立自由民主政体。打碎之后建立的自由民主政体,作为后事之师不难发现其解决问题效力匮乏。在巴以冲突的当下,如何解决犹太人问题这一历史悠长的时代难题,也亟须学术界作出更多的研究与贡献。

生物学批判性思维测评试题的解析

围绕生物学知识与批判性思维目标,通过引用原题、整合改编、自主编制试题,分析生物学批判性思维测评试题,为批判性思维在生物学教学中的应用提供思路,为生物学批判性思维试题的开发研究提供参考。

基于结构性问题培养全科医生深度思考能力的方法研究

背景住院医师规范化培训的全科学员在各临床专科轮转时间短暂,学习内容宽泛,需要增强学习自主性。全科师资会面临不同背景学员,如5+3、转岗、专业硕士,或3+2助理全科、实习医生、公共卫生学员等,如何在培训中,使不同培训对象能各取所需,达到应有培训效果,需要认真研究。目的探索基于结构性问题的培训方式,提高学员学习主动性,培养其深度思考能力的作用。方法选取2020年江苏省全科/助理全科骨干师资培训班(骨干师资班)和基层卫生人才能力提升培训班(基层人才班)的学员作为研究对象。在每次学习活动结束,即刻组织学员讨论,要求依次回答“通过学习1.你学到了什么?2.还有哪些疑问?3.既往有什么相同或者类似的经验与大家分享?4.对今后工作的启发?”根据上述问题思路自行设计调查问卷调查培训学员对结构性问题培训方式的认同度,分为第1部分一般资料,包括性别、学历、职称、工作单位、岗位、工作年限、参与培训的项目等;第2部分是对“问题1、问题2、问题3、问题4和培训形式5”的解释性问题,5个维度共计20个条目,备选项是四等级,赋值为“1=非常同意,2=同意,3=不太同意,4=完全不同意”。结果培训对象对所有条目选择同意与非常同意的百分比均大于95%。不同性别、年龄、职称、岗位、工作年限培训学员等对结构性问题培训方法的认同程度比较,差异无统计学意义(P>0.05);不同学历、工作单位和培训项目培训学员对结构性问题培训方法的认同程度比较,差异有统计学意义(P<0.05)。相较于研究生及以上学历,本科培训对象对问题3、问题4的认同程度更高(P<0.05)。相较于三级医院,城市社区卫生服务中心(乡镇卫生院)和二级医院的培训对象对于问题1~4认同程度更高(P<0.05)。相较于骨干师资班,基层人才班的培训对象对于问题1~4及培训形式5认同程度更高(P<0.05)。结论培训实践中总结出的四个开放式问题,内容简单,含义递进,具有内在联系。运用结构性问题培训全科学员,形式灵活,能够激发被培训者的深度思考,该方法在基层卫生人才培训班获得更高的认可程度,表明适用于全科医生的培养。