PAL-BERT:An Improved Question Answering Model

In the field of natural language processing(NLP),there have been various pre-training language models in recent years,with question answering systems gaining significant attention.However,as algorithms,data,and computing power advance,the issue of increasingly larger models and a growing number of parameters has surfaced.Consequently,model training has become more costly and less efficient.To enhance the efficiency and accuracy of the training process while reducing themodel volume,this paper proposes a first-order pruningmodel PAL-BERT based on the ALBERT model according to the characteristics of question-answering(QA)system and language model.Firstly,a first-order network pruning method based on the ALBERT model is designed,and the PAL-BERT model is formed.Then,the parameter optimization strategy of the PAL-BERT model is formulated,and the Mish function was used as an activation function instead of ReLU to improve the performance.Finally,after comparison experiments with traditional deep learning models TextCNN and BiLSTM,it is confirmed that PALBERT is a pruning model compression method that can significantly reduce training time and optimize training efficiency.Compared with traditional models,PAL-BERT significantly improves the NLP task’s performance.

DPAL-BERT:A Faster and Lighter Question Answering Model

Recent advancements in natural language processing have given rise to numerous pre-training language models in question-answering systems.However,with the constant evolution of algorithms,data,and computing power,the increasing size and complexity of these models have led to increased training costs and reduced efficiency.This study aims to minimize the inference time of such models while maintaining computational performance.It also proposes a novel Distillation model for PAL-BERT(DPAL-BERT),specifically,employs knowledge distillation,using the PAL-BERT model as the teacher model to train two student models:DPAL-BERT-Bi and DPAL-BERTC.This research enhances the dataset through techniques such as masking,replacement,and n-gram sampling to optimize knowledge transfer.The experimental results showed that the distilled models greatly outperform models trained from scratch.In addition,although the distilled models exhibit a slight decrease in performance compared to PAL-BERT,they significantly reduce inference time to just 0.25%of the original.This demonstrates the effectiveness of the proposed approach in balancing model performance and efficiency.

Operational requirements analysis method based on question answering of WEKG

The weapon and equipment operational requirement analysis(WEORA) is a necessary condition to win a future war,among which the acquisition of knowledge about weapons and equipment is a great challenge. The main challenge is that the existing weapons and equipment data fails to carry out structured knowledge representation, and knowledge navigation based on natural language cannot efficiently support the WEORA. To solve above problem, this research proposes a method based on question answering(QA) of weapons and equipment knowledge graph(WEKG) to construct and navigate the knowledge related to weapons and equipment in the WEORA. This method firstly constructs the WEKG, and builds a neutral network-based QA system over the WEKG by means of semantic parsing for knowledge navigation. Finally, the method is evaluated and a chatbot on the QA system is developed for the WEORA. Our proposed method has good performance in the accuracy and efficiency of searching target knowledge, and can well assist the WEORA.

MKEAH:Multimodal knowledge extraction and accumulation based on hyperplane embedding for knowledge-based visual question answering

Background External knowledge representations play an essential role in knowledge-based visual question and answering to better understand complex scenarios in the open world.Recent entity-relationship embedding approaches are deficient in representing some complex relations,resulting in a lack of topic-related knowledge and redundancy in topic-irrelevant information.Methods To this end,we propose MKEAH:Multimodal Knowledge Extraction and Accumulation on Hyperplanes.To ensure that the lengths of the feature vectors projected onto the hyperplane compare equally and to filter out sufficient topic-irrelevant information,two losses are proposed to learn the triplet representations from the complementary views:range loss and orthogonal loss.To interpret the capability of extracting topic-related knowledge,we present the Topic Similarity(TS)between topic and entity-relations.Results Experimental results demonstrate the effectiveness of hyperplane embedding for knowledge representation in knowledge-based visual question answering.Our model outperformed state-of-the-art methods by 2.12%and 3.24%on two challenging knowledge-request datasets:OK-VQA and KRVQA,respectively.Conclusions The obvious advantages of our model in TS show that using hyperplane embedding to represent multimodal knowledge can improve its ability to extract topic-related knowledge.

Question classification in question answering based on real-world web data sets

To improve question answering （QA） performance based on real-world web data sets,a new set of question classes and a general answer re-ranking model are defined.With pre-defined dictionary and grammatical analysis,the question classifier draws both semantic and grammatical information into information retrieval and machine learning methods in the form of various training features,including the question word,the main verb of the question,the dependency structure,the position of the main auxiliary verb,the main noun of the question,the top hypernym of the main noun,etc.Then the QA query results are re-ranked by question class information.Experiments show that the questions in real-world web data sets can be accurately classified by the classifier,and the QA results after re-ranking can be obviously improved.It is proved that with both semantic and grammatical information,applications such as QA, built upon real-world web data sets, can be improved,thus showing better performance.

Influencing factors of answer adoption in social Q&A communities from users' perspective: Taking Zhihu as an example

Purpose: Taking Zhihu as the object for a case study, we intend to analyze the key factors that have affected users on adopting answers in social Q&A(SQA) websites.Design/methodology/approach: With information adoption model(IAM) as the theoretical foundation and widely accepted evaluation criteria for answer quality in SQA sites as variables, we constructed a factor model that has influenced SQA community users to adopt offered answers. With the partial least squares(PLS) technique, our model was then empirically tested through a sample of 311 Zhihu users.Findings: Our results showed that answer usefulness is the most effective variable, and answer interactivity and answer entertainment both have positive and significant impacts on users’ attitude to adopt answers in an SQA community. Except for novelty, other three components of answer quality, i.e. knowledge, reliability, and solution to the problem have all significant effect on answer usefulness.Research limitations: First, due to the limited sample size, it is still questionable if our research results based on Zhihu could be applied to other SQA communities. Second, our questionnaires were mainly designed to investigate how users felt about the answers in an SQA site, but did not differentiate the content of the answer itself.Practical implications: As a three-year-old SQA platform, Zhihu has developed very quickly with its high-quality answers and public intellectual users, and has been regarded as one of the representatives of fast emerging Chinese SQA communities in recent years. Our studycould help shed light on users’ information sharing and knowledge adoption behaviors in a Chinese SQA site, such as Zhihu. Originality/value: Compared with previous studies on answer quality assessments in SQA sites and on information adoption model, to the best of our knowledge, this is one of the pioneer studies which combined answer qualities with users’ intention of adopting SQA answers. Our study on user answer adoption in Zhihu community could further develop the theory of IAM. This study showed that answer usefulness is the most important motivation of Zhihu users in the process of adopting answers.

ANSWER2000在小流域土壤侵蚀过程模拟中的应用研究

ANSWERS2 0 0 0是一个用于流域土壤侵蚀过程模拟的分散型物理模型 ,将此模型运用于三峡库区小流域侵蚀产沙、地表径流以及不同土地利用类型水沙分布状况的模拟中。通过两个不同小流域模拟结果的对比 ,采用误差百分比、线性回归以及 Nash- Sutcliffe效率 3种方法 ,分析和评价了模型的模拟效果。结果表明 ,模型在应用于我国三峡库区小流域土壤侵蚀模拟时 ,其模拟结果与实测结果具有较高的吻合度 ,模拟结果基本可信。但是 ,对于一些陡坡林地等特殊地类 ,模型的模拟误差较大 ,其模拟精度还有待于进一步提高。

ANSWERS模型及其应用

ANSWERS模型主要是针对欧洲平原地区研发的分散型物理模型。介绍了模型的研发历史、结构、输入和输出信息以及模型的应用。ANSWERS主要适用于缓坡地形区的径流模拟、侵蚀模拟和农业污染物运移模拟。如何根据中国的实际合理确定模型参数,使模型在我国复杂地形区应用,尚有许多问题需要研究。

土壤侵蚀建模中ANSWERS及地理信息系统ARC/INFO^R的应用研究

研究了土壤侵蚀模型ＡＮＳＷＥＲＳ和地理信息系统（ＧＩＳ）ＡＲＣ／ＩＮＦＯ之间的连结。采用ＡＲＣ／ＩＮＦＯ建立数据库和ＡＮＳＷＥＲＳ进行实际操作，加强了该模型在制定水保措施中的应用。同时，研究出的ＡＲＣＡＮＳ模型，使ＡＲＣ／ＩＮＦＯ和ＡＮＳＷＥＲＳ之间的连结更为容易、有效。最后，对四川紫色丘陵区的一个小流域实施了模拟，以展示连结情况和一些值得注意的问题。

A Novel Bidirectional LSTM and Attention Mechanism Based Neural Network for Answer Selection in Community Question Answering

Deep learning models have been shown to have great advantages in answer selection tasks.The existing models,which employ encoder-decoder recurrent neural network(RNN),have been demonstrated to be effective.However,the traditional RNN-based models still suffer from limitations such as 1)high-dimensional data representation in natural language processing and 2)biased attentive weights for subsequent words in traditional time series models.In this study,a new answer selection model is proposed based on the Bidirectional Long Short-Term Memory(Bi-LSTM)and attention mechanism.The proposed model is able to generate the more effective question-answer pair representation.Experiments on a question answering dataset that includes information from multiple fields show the great advantages of our proposed model.Specifically,we achieve a maximum improvement of 3.8%over the classical LSTM model in terms of mean average precision.

Analysis of community question-answering issues via machine learning and deep learning:State-of-the-art review

Over the last couple of decades,community question-answering sites(CQAs)have been a topic of much academic interest.Scholars have often leveraged traditional machine learning(ML)and deep learning(DL)to explore the ever-growing volume of content that CQAs engender.To clarify the current state of the CQA literature that has used ML and DL,this paper reports a systematic literature review.The goal is to summarise and synthesise the major themes of CQA research related to(i)questions,(ii)answers and(iii)users.The final review included 133 articles.Dominant research themes include question quality,answer quality,and expert identification.In terms of dataset,some of the most widely studied platforms include Yahoo!Answers,Stack Exchange and Stack Overflow.The scope of most articles was confined to just one platform with few cross-platform investigations.Articles with ML outnumber those with DL.Nonetheless,the use of DL in CQA research is on an upward trajectory.A number of research directions are proposed.

借鉴Google Answers构建高校图书馆咨询专家队伍

分析Google Answers,借鉴它的问答模式、用人政策等,从而构想依托高校专家教授的人力资源来建立高校图书馆的咨询专家队伍。

ACLSTM:A Novel Method for CQA Answer Quality Prediction Based on Question-Answer Joint Learning

Given the limitations of the community question answering(CQA)answer quality prediction method in measuring the semantic information of the answer text,this paper proposes an answer quality prediction model based on the question-answer joint learning(ACLSTM).The attention mechanism is used to obtain the dependency relationship between the Question-and-Answer(Q&A)pairs.Convolutional Neural Network(CNN)and Long Short-term Memory Network(LSTM)are used to extract semantic features of Q&A pairs and calculate their matching degree.Besides,answer semantic representation is combined with other effective extended features as the input representation of the fully connected layer.Compared with other quality prediction models,the ACLSTM model can effectively improve the prediction effect of answer quality.In particular,the mediumquality answer prediction,and its prediction effect is improved after adding effective extended features.Experiments prove that after the ACLSTM model learning,the Q&A pairs can better measure the semantic match between each other,fully reflecting the model’s superior performance in the semantic information processing of the answer text.

A multi-attention RNN-based relation linking approach for question answering over knowledge base

Aiming at the relation linking task for question answering over knowledge base,especially the multi relation linking task for complex questions,a relation linking approach based on the multi-attention recurrent neural network(RNN)model is proposed,which works for both simple and complex questions.First,the vector representations of questions are learned by the bidirectional long short-term memory(Bi-LSTM)model at the word and character levels,and named entities in questions are labeled by the conditional random field(CRF)model.Candidate entities are generated based on a dictionary,the disambiguation of candidate entities is realized based on predefined rules,and named entities mentioned in questions are linked to entities in knowledge base.Next,questions are classified into simple or complex questions by the machine learning method.Starting from the identified entities,for simple questions,one-hop relations are collected in the knowledge base as candidate relations;for complex questions,two-hop relations are collected as candidates.Finally,the multi-attention Bi-LSTM model is used to encode questions and candidate relations,compare their similarity,and return the candidate relation with the highest similarity as the result of relation linking.It is worth noting that the Bi-LSTM model with one attentions is adopted for simple questions,and the Bi-LSTM model with two attentions is adopted for complex questions.The experimental results show that,based on the effective entity linking method,the Bi-LSTM model with the attention mechanism improves the relation linking effectiveness of both simple and complex questions,which outperforms the existing relation linking methods based on graph algorithm or linguistics understanding.

Designing an automated FAQ answering system for farmers based on hybrid strategies

Purpose: The purpose of this study is to develop an automated frequently asked question(FAQ) answering system for farmers. This paper presents an approach for calculating the similarity between Chinese sentences based on hybrid strategies.Design/methodology/approach: We analyzed the factors influencing the successful matching between a user's question and a question-answer(QA) pair in the FAQ database. Our approach is based on a combination of multiple factors. Experiments were conducted to test the performance of our method.Findings: Experiments show that this proposed method has higher accuracy. Compared with similarity calculation based on TF-IDF,the sentence surface forms and the semantic relations,the proposed method based on hybrid strategies has a superior performance in precision,recall and F-measure value.Research limitations: The FAQ answering system is only capable of meeting users' demand for text retrieval at present. In the future,the system needs to be improved to meet users' demand for retrieving images and videos.Practical implications: This FAQ answering system will help farmers utilize agricultural information resources more efficiently.Originality/value: We design the algorithms for calculating similarity of Chinese sentences based on hybrid strategies,which integrate the question surface similarity,the question semantic similarity and the question-answer similarity based on latent semantic analysis(LSA) to find answers to a user's question.

A novel approach for agent ontology and its application in question answering

The information integration method of semantic web based on agent ontology(SWAO method) was put forward aiming at the problems in current network environment,which integrates,analyzes and processes enormous web information and extracts answers on the basis of semantics. With SWAO method as the clue,the following technologies were studied:the method of concept extraction based on semantic term mining,agent ontology construction method on account of multi-points and the answer extraction in view of semantic inference. Meanwhile,the structural model of the question answering system applying ontology was presented,which adopts OWL language to describe domain knowledge from where QA system infers and extracts answers by Jena inference engine. In the system testing,the precision rate reaches 86%,and the recalling rate is 93%. The experimental results prove that it is feasible to use the method to develop a question answering system,which is valuable for further study in more depth.

评短片《Answer》:如果精神被取代了,我们何以为人?

《Answer》是一部略带魔幻现实风格的短片,剧情聚焦于一个有着手机依赖症的大学生,而主题实际指向现代科技的发展与传统伦理、人性美好的背离。

The Answer is 9 or 10?

Below is Problem 9 in Singapore MathematicalOlympiad 2001:Five people jointly bought a lottery ticketwhich won the first prize.They decided to keeptheir ticket in a locker installed with a number oflocks.Each person may hold the keys to more thanone lock.What is the minimum number of locks nee-ded to serve the purpose that any 3 together will beable to open the locker but any 2 of them togetherwill not be able to open it ?A.6 B.9 C.10 D.

非点源污染模型ANSWERS-2000的水文子模型研究

非点源污染的前期过程，是在流域面上污染物质或是营养元素随着降雨径流的产生而产生。利用实验室大型土槽试验，研究ANSWERS-2000模型的水文子模型对于人工降雨事件的模拟精度，探索ANSWERS-2000在理想坡面上的适用情况，并且尽可能准确地给出其水文模型的部分参数的取值。经过计算发现该水文子模型模拟理想坡面的误差较小，但是存在系统偏大的情况；整个计算单元的Manning糙率取0．03～0．07。存在的问题有参数土表面结皮层厚度只能定性无法准确的定量描述；降雨装置测流装置对于径流模拟产生的影响较大。