PAL-BERT:An Improved Question Answering Model

In the field of natural language processing(NLP),there have been various pre-training language models in recent years,with question answering systems gaining significant attention.However,as algorithms,data,and computing power advance,the issue of increasingly larger models and a growing number of parameters has surfaced.Consequently,model training has become more costly and less efficient.To enhance the efficiency and accuracy of the training process while reducing themodel volume,this paper proposes a first-order pruningmodel PAL-BERT based on the ALBERT model according to the characteristics of question-answering(QA)system and language model.Firstly,a first-order network pruning method based on the ALBERT model is designed,and the PAL-BERT model is formed.Then,the parameter optimization strategy of the PAL-BERT model is formulated,and the Mish function was used as an activation function instead of ReLU to improve the performance.Finally,after comparison experiments with traditional deep learning models TextCNN and BiLSTM,it is confirmed that PALBERT is a pruning model compression method that can significantly reduce training time and optimize training efficiency.Compared with traditional models,PAL-BERT significantly improves the NLP task’s performance.

DPAL-BERT:A Faster and Lighter Question Answering Model

Recent advancements in natural language processing have given rise to numerous pre-training language models in question-answering systems.However,with the constant evolution of algorithms,data,and computing power,the increasing size and complexity of these models have led to increased training costs and reduced efficiency.This study aims to minimize the inference time of such models while maintaining computational performance.It also proposes a novel Distillation model for PAL-BERT(DPAL-BERT),specifically,employs knowledge distillation,using the PAL-BERT model as the teacher model to train two student models:DPAL-BERT-Bi and DPAL-BERTC.This research enhances the dataset through techniques such as masking,replacement,and n-gram sampling to optimize knowledge transfer.The experimental results showed that the distilled models greatly outperform models trained from scratch.In addition,although the distilled models exhibit a slight decrease in performance compared to PAL-BERT,they significantly reduce inference time to just 0.25%of the original.This demonstrates the effectiveness of the proposed approach in balancing model performance and efficiency.

Operational requirements analysis method based on question answering of WEKG

The weapon and equipment operational requirement analysis(WEORA) is a necessary condition to win a future war,among which the acquisition of knowledge about weapons and equipment is a great challenge. The main challenge is that the existing weapons and equipment data fails to carry out structured knowledge representation, and knowledge navigation based on natural language cannot efficiently support the WEORA. To solve above problem, this research proposes a method based on question answering(QA) of weapons and equipment knowledge graph(WEKG) to construct and navigate the knowledge related to weapons and equipment in the WEORA. This method firstly constructs the WEKG, and builds a neutral network-based QA system over the WEKG by means of semantic parsing for knowledge navigation. Finally, the method is evaluated and a chatbot on the QA system is developed for the WEORA. Our proposed method has good performance in the accuracy and efficiency of searching target knowledge, and can well assist the WEORA.

Combining Medical Care with Elderly Care

With the continuous expansion of the demand in China for the integration of medical care and elderly care,more social capital will be directed into this field.A LTHOUGHT answers to the question“What is happiness?”may vary among young people,for most senior citizens the answer is by and large the same:to be looked after properly.

职面试小技巧

Prepare well before you go to a job interview.First,understand the company's goals.Then it will be easierto answer questions about them.Second,learn about the job.Third,practice answering common intervie wquestions.Fourth,wear nice clothes and arriveat your interview on time.And after the interview.

ANSWER模型评估新疆咸水灌溉棉花产量与效益

利用咸水或微咸水进行农田灌溉是缓解中国新疆地区农业水资源供需矛盾从而保障当地棉花产业可持续发展的主要途径之一。为了明确不同咸水灌溉措施对棉花产量及经济效益的影响,该研究通过2 a的棉花膜下滴灌大田试验和文献检索获取了新疆9个不同试验地点的土壤、作物及灌溉等数据资料,评估作物产量-水盐胁迫响应分析模型(ANalytical Salt WatER,ANSWER)在新疆棉花产量评估中的适用性和可靠性,并结合经济收支平衡方法,模拟分析不同咸水灌溉措施(包括不同灌溉定额和灌溉水电导率的组合)对棉花产量与经济效益的影响。采用决定系数(R2)、均方根误差(root mean squared error,RMSE)、相对均方根误差(relative root mean squared error,RRMSE)评价模型精度。结果表明,在9个不同试验地点,ANSWER模型均可较准确地估算棉花的相对产量,其估算值与实测值之间的R^(2)≥0.54,RMSE≤0.14,RRMSE≤0.16;不同试验地点,优化获得的各个模型生物参数(与棉花根系吸水的水盐胁迫响应相关的参数)差异较小,变异系数的绝对值处于0.08~0.37之间;基于不同试验地点优化的各生物参数均值估算各地的棉花相对产量,其与实测值仍然吻合良好(R^(2)为0.59,RMSE为0.06,RRMSE为0.07);此外,当灌溉水电导率一定时,棉花净收益随灌溉定额增加呈先增后降的趋势,净收益达到峰值所需的灌溉定额随灌溉水电导率升高而迅速增加;当灌溉水电导率不大于10 dS/m时,通过加大供水量均可获得与淡水灌溉相当的净收益。研究可为新疆地区棉花产量与效益评估以及咸水资源合理开发利用提供理论依据。

Analysis of community question-answering issues via machine learning and deep learning:State-of-the-art review

Over the last couple of decades,community question-answering sites(CQAs)have been a topic of much academic interest.Scholars have often leveraged traditional machine learning(ML)and deep learning(DL)to explore the ever-growing volume of content that CQAs engender.To clarify the current state of the CQA literature that has used ML and DL,this paper reports a systematic literature review.The goal is to summarise and synthesise the major themes of CQA research related to(i)questions,(ii)answers and(iii)users.The final review included 133 articles.Dominant research themes include question quality,answer quality,and expert identification.In terms of dataset,some of the most widely studied platforms include Yahoo!Answers,Stack Exchange and Stack Overflow.The scope of most articles was confined to just one platform with few cross-platform investigations.Articles with ML outnumber those with DL.Nonetheless,the use of DL in CQA research is on an upward trajectory.A number of research directions are proposed.

Expert Recommendation in Community Question Answering via Heterogeneous Content Network Embedding

ExpertRecommendation(ER)aims to identify domain experts with high expertise and willingness to provide answers to questions in Community Question Answering(CQA)web services.How to model questions and users in the heterogeneous content network is critical to this task.Most traditional methods focus on modeling questions and users based on the textual content left in the community while ignoring the structural properties of heterogeneous CQA networks and always suffering from textual data sparsity issues.Recent approaches take advantage of structural proximities between nodes and attempt to fuse the textual content of nodes for modeling.However,they often fail to distinguish the nodes’personalized preferences and only consider the textual content of a part of the nodes in network embedding learning,while ignoring the semantic relevance of nodes.In this paper,we propose a novel framework that jointly considers the structural proximity relations and textual semantic relevance to model users and questions more comprehensively.Specifically,we learn topology-based embeddings through a hierarchical attentive network learning strategy,in which the proximity information and the personalized preference of nodes are encoded and preserved.Meanwhile,we utilize the node’s textual content and the text correlation between adjacent nodes to build the content-based embedding through a meta-context-aware skip-gram model.In addition,the user’s relative answer quality is incorporated to promote the ranking performance.Experimental results show that our proposed framework consistently and significantly outperforms the state-of-the-art baselines on three real-world datasets by taking the deep semantic understanding and structural feature learning together.The performance of the proposed work is analyzed in terms of MRR,P@K,and MAP and is proven to be more advanced than the existing methodologies.

ALBERT with Knowledge Graph Encoder Utilizing Semantic Similarity for Commonsense Question Answering

Recently,pre-trained language representation models such as bidirec-tional encoder representations from transformers(BERT)have been performing well in commonsense question answering(CSQA).However,there is a problem that the models do not directly use explicit information of knowledge sources existing outside.To augment this,additional methods such as knowledge-aware graph network(KagNet)and multi-hop graph relation network(MHGRN)have been proposed.In this study,we propose to use the latest pre-trained language model a lite bidirectional encoder representations from transformers(ALBERT)with knowledge graph information extraction technique.We also propose to applying the novel method,schema graph expansion to recent language models.Then,we analyze the effect of applying knowledge graph-based knowledge extraction techniques to recent pre-trained language models and confirm that schema graph expansion is effective in some extent.Furthermore,we show that our proposed model can achieve better performance than existing KagNet and MHGRN models in CommonsenseQA dataset.

Information Extraction Based on Multi-turn Question Answering for Analyzing Korean Research Trends

Analyzing Research and Development(R&D)trends is important because it can influence future decisions regarding R&D direction.In typical trend analysis,topic or technology taxonomies are employed to compute the popularities of the topics or codes over time.Although it is simple and effective,the taxonomies are difficult to manage because new technologies are introduced rapidly.Therefore,recent studies exploit deep learning to extract pre-defined targets such as problems and solutions.Based on the recent advances in question answering(QA)using deep learning,we adopt a multi-turn QA model to extract problems and solutions from Korean R&D reports.With the previous research,we use the reports directly and analyze the difficulties in handling them using QA style on Information Extraction(IE)for sentence-level benchmark dataset.After investigating the characteristics of Korean R&D,we propose a model to deal with multiple and repeated appearances of targets in the reports.Accordingly,we propose a model that includes an algorithm with two novel modules and a prompt.A newly proposed methodology focuses on reformulating a question without a static template or pre-defined knowledge.We show the effectiveness of the proposed model using a Korean R&D report dataset that we constructed and presented an in-depth analysis of the benefits of the multi-turn QA model.

Improved Blending Attention Mechanism in Visual Question Answering

Visual question answering(VQA)has attracted more and more attention in computer vision and natural language processing.Scholars are committed to studying how to better integrate image features and text features to achieve better results in VQA tasks.Analysis of all features may cause information redundancy and heavy computational burden.Attention mechanism is a wise way to solve this problem.However,using single attention mechanism may cause incomplete concern of features.This paper improves the attention mechanism method and proposes a hybrid attention mechanism that combines the spatial attention mechanism method and the channel attention mechanism method.In the case that the attention mechanism will cause the loss of the original features,a small portion of image features were added as compensation.For the attention mechanism of text features,a selfattention mechanism was introduced,and the internal structural features of sentences were strengthened to improve the overall model.The results show that attention mechanism and feature compensation add 6.1%accuracy to multimodal low-rank bilinear pooling network.

Deep Multi-Module Based Language Priors Mitigation Model for Visual Question Answering

The original intention of visual question answering(VQA)models is to infer the answer based on the relevant information of the question text in the visual image,but many VQA models often yield answers that are biased by some prior knowledge,especially the language priors.This paper proposes a mitigation model called language priors mitigation-VQA(LPM-VQA)for the language priors problem in VQA model,which divides language priors into positive and negative language priors.Different network branches are used to capture and process the different priors to achieve the purpose of mitigating language priors.A dynamically-changing language prior feedback objective function is designed with the intermediate results of some modules in the VQA model.The weight of the loss value for each answer is dynamically set according to the strength of its language priors to balance its proportion in the total VQA loss to further mitigate the language priors.This model does not depend on the baseline VQA architectures and can be configured like a plug-in to improve the performance of the model over most existing VQA models.The experimental results show that the proposed model is general and effective,achieving state-of-the-art accuracy in the VQA-CP v2 dataset.

Valuing the Core Values

Wisdom and Belief of the East-Understanding the Chinese Values Author:Han Zhen Price:RMB 95 Paperback,207 pages Published by Foreign Languages Press China is increasingly taking center stage in the international arena,and people from around the world are particularly interested in the path the country has taken and the direction it is heading in.More importantly,they want to know what China can contribute to the world.All this curiosity may be boiled down to a seemingly simple question:How to understand China?However,it is not a question that is easily answered in just a few words.

一种基于事件的Web服务组合方法

为获得一种既易于实现又能满足用户多样化需求的服务组合的有效途径,提出一种基于事件的服务组合方法.首先定义了一种基于ECA(event-condition-action)规则的语言——简单服务事件语言.在这种语言基础上,通过模块化方法构造的用于描述组合服务的组合方案,不但解决了采用AI规划(artificial intelligent planning)时服务组合域表示困难的问题,而且解决了采用UML(unified modeling language)等技术时描述能力不足的问题.随后,为有效地表示组合方案,完成了它的语义定义以及answer set程序编码工作.最后利用answer set编程(answer set programming)技术实现了对组合轨迹的表示.

一种支持个性化协调的服务机器人体系结构

本文提出一种支持个性化协调的服务机器人体系结构(individualized coordination architecture,ICA).主要动机是通过自然语言人机对话获取用户的个人特性和其他信息,通过对这些信息进行自动推理和规划,实现利用个人特性的自动问题求解,并满足家庭环境对服务机器人的应用要求.本文着重介绍ICA的主要部件的功能及其相互衔接方式,描述任务规划的机制和实现手段,并通过一个实例说明在一个初步实现的原型系统中从自然语言到任务规划的完整过程.

ANSWER2000在小流域土壤侵蚀过程模拟中的应用研究

ANSWERS2 0 0 0是一个用于流域土壤侵蚀过程模拟的分散型物理模型 ,将此模型运用于三峡库区小流域侵蚀产沙、地表径流以及不同土地利用类型水沙分布状况的模拟中。通过两个不同小流域模拟结果的对比 ,采用误差百分比、线性回归以及 Nash- Sutcliffe效率 3种方法 ,分析和评价了模型的模拟效果。结果表明 ,模型在应用于我国三峡库区小流域土壤侵蚀模拟时 ,其模拟结果与实测结果具有较高的吻合度 ,模拟结果基本可信。但是 ,对于一些陡坡林地等特殊地类 ,模型的模拟误差较大 ,其模拟精度还有待于进一步提高。

ANSWERS模型及其应用

ANSWERS模型主要是针对欧洲平原地区研发的分散型物理模型。介绍了模型的研发历史、结构、输入和输出信息以及模型的应用。ANSWERS主要适用于缓坡地形区的径流模拟、侵蚀模拟和农业污染物运移模拟。如何根据中国的实际合理确定模型参数,使模型在我国复杂地形区应用,尚有许多问题需要研究。

ER模型的逻辑表示途径

利用Answer set编程表示ER模型,从而为ER模型提供了一种新颖的逻辑表示途径。首先,完成ER模式的语法与语义定义;其次,利用Answer set编程实现ER模式的逻辑编程表示,并且这里的编程可自动实现;最后,完成以上表示的合理性证明。工作不仅克服了ER模型作为图形化工具的缺陷,使得它具有了自动推理能力,而且也为利用ER模型实现异构数据库之间的语义协作奠定了理论基础。

土壤侵蚀建模中ANSWERS及地理信息系统ARC/INFO^R的应用研究

研究了土壤侵蚀模型ＡＮＳＷＥＲＳ和地理信息系统（ＧＩＳ）ＡＲＣ／ＩＮＦＯ之间的连结。采用ＡＲＣ／ＩＮＦＯ建立数据库和ＡＮＳＷＥＲＳ进行实际操作，加强了该模型在制定水保措施中的应用。同时，研究出的ＡＲＣＡＮＳ模型，使ＡＲＣ／ＩＮＦＯ和ＡＮＳＷＥＲＳ之间的连结更为容易、有效。最后，对四川紫色丘陵区的一个小流域实施了模拟，以展示连结情况和一些值得注意的问题。