Network Configuration Entity Extraction Method Based on Transformer with Multi-Head Attention Mechanism

Nowadays,ensuring thequality of networkserviceshas become increasingly vital.Experts are turning toknowledge graph technology,with a significant emphasis on entity extraction in the identification of device configurations.This research paper presents a novel entity extraction method that leverages a combination of active learning and attention mechanisms.Initially,an improved active learning approach is employed to select the most valuable unlabeled samples,which are subsequently submitted for expert labeling.This approach successfully addresses the problems of isolated points and sample redundancy within the network configuration sample set.Then the labeled samples are utilized to train the model for network configuration entity extraction.Furthermore,the multi-head self-attention of the transformer model is enhanced by introducing the Adaptive Weighting method based on the Laplace mixture distribution.This enhancement enables the transformer model to dynamically adapt its focus to words in various positions,displaying exceptional adaptability to abnormal data and further elevating the accuracy of the proposed model.Through comparisons with Random Sampling(RANDOM),Maximum Normalized Log-Probability(MNLP),Least Confidence(LC),Token Entrop(TE),and Entropy Query by Bagging(EQB),the proposed method,Entropy Query by Bagging and Maximum Influence Active Learning(EQBMIAL),achieves comparable performance with only 40% of the samples on both datasets,while other algorithms require 50% of the samples.Furthermore,the entity extraction algorithm with the Adaptive Weighted Multi-head Attention mechanism(AW-MHA)is compared with BILSTM-CRF,Mutil_Attention-Bilstm-Crf,Deep_Neural_Model_NER and BERT_Transformer,achieving precision rates of 75.98% and 98.32% on the two datasets,respectively.Statistical tests demonstrate the statistical significance and effectiveness of the proposed algorithms in this paper.

MEIM:A Multi-Source Software Knowledge Entity Extraction Integration Model

Entity recognition and extraction are the foundations of knowledge graph construction.Entity data in the field of software engineering come from different platforms and communities,and have different formats.This paper divides multi-source software knowledge entities into unstructured data,semi-structured data and code data.For these different types of data,Bi-directional Long Short-Term Memory(Bi-LSTM)with Conditional Random Field(CRF),template matching,and abstract syntax tree are used and integrated into a multi-source software knowledge entity extraction integration model(MEIM)to extract software entities.The model can be updated continuously based on user’s feedbacks to improve the accuracy.To deal with the shortage of entity annotation datasets,keyword extraction methods based on Term Frequency–Inverse Document Frequency(TF-IDF),TextRank,and K-Means are applied to annotate tasks.The proposed MEIM model is applied to the Spring Boot framework,which demonstrates good adaptability.The extracted entities are used to construct a knowledge graph,which is applied to association retrieval and association visualization.

The Entity Relationship Extraction Method Using Improved RoBERTa and Multi-Task Learning

There is a growing amount of data uploaded to the internet every day and it is important to understand the volume of those data to find a better scheme to process them.However,the volume of internet data is beyond the processing capabilities of the current internet infrastructure.Therefore,engineering works using technology to organize and analyze information and extract useful information are interesting in both industry and academia.The goal of this paper is to explore the entity relationship based on deep learning,introduce semantic knowledge by using the prepared language model,develop an advanced entity relationship information extraction method by combining Robustly Optimized BERT Approach(RoBERTa)and multi-task learning,and combine the intelligent characters in the field of linguistic,called Robustly Optimized BERT Approach+Multi-Task Learning(RoBERTa+MTL).To improve the effectiveness of model interaction,multi-task teaching is used to implement the observation information of auxiliary tasks.Experimental results show that our method has achieved an accuracy of 88.95 entity relationship extraction,and a further it has achieved 86.35%of accuracy after being combined with multi-task learning.

Corpus of Carbonate Platforms with Lexical Annotations for Named Entity Recognition

An obviously challenging problem in named entity recognition is the construction of the kind data set of entities.Although some research has been conducted on entity database construction,the majority of them are directed at Wikipedia or the minority at structured entities such as people,locations and organizational nouns in the news.This paper focuses on the identification of scientific entities in carbonate platforms in English literature,using the example of carbonate platforms in sedimentology.Firstly,based on the fact that the reasons for writing literature in key disciplines are likely to be provided by multidisciplinary experts,this paper designs a literature content extraction method that allows dealing with complex text structures.Secondly,based on the literature extraction content,we formalize the entity extraction task(lexicon and lexical-based entity extraction)for entity extraction.Furthermore,for testing the accuracy of entity extraction,three currently popular recognition methods are chosen to perform entity detection in this paper.Experiments show that the entity data set provided by the lexicon and lexical-based entity extraction method is of significant assistance for the named entity recognition task.This study presents a pilot study of entity extraction,which involves the use of a complex structure and specialized literature on carbonate platforms in English.

Combining Deep Learning with Knowledge Graph for Design Knowledge Acquisition in Conceptual Product Design

The acquisition of valuable design knowledge from massive fragmentary data is challenging for designers in conceptual product design.This study proposes a novel method for acquiring design knowledge by combining deep learning with knowledge graph.Specifically,the design knowledge acquisition method utilises the knowledge extraction model to extract design-related entities and relations from fragmentary data,and further constructs the knowledge graph to support design knowledge acquisition for conceptual product design.Moreover,the knowledge extraction model introduces ALBERT to solve memory limitation and communication overhead in the entity extraction module,and uses multi-granularity information to overcome segmentation errors and polysemy ambiguity in the relation extraction module.Experimental comparison verified the effectiveness and accuracy of the proposed knowledge extraction model.The case study demonstrated the feasibility of the knowledge graph construction with real fragmentary porcelain data and showed the capability to provide designers with interconnected and visualised design knowledge.

Entity and relation extraction with rule-guided dictionary as domain knowledge

Entity and relation extraction is an indispensable part of domain knowledge graph construction,which can serve relevant knowledge needs in a specific domain,such as providing support for product research,sales,risk control,and domain hotspot analysis.The existing entity and relation extraction methods that depend on pretrained models have shown promising performance on open datasets.However,the performance of these methods degrades when they face domain-specific datasets.Entity extraction models treat characters as basic semantic units while ignoring known character dependency in specific domains.Relation extraction is based on the hypothesis that the relations hidden in sentences are unified,thereby neglecting that relations may be diverse in different entity tuples.To address the problems above,this paper first introduced prior knowledge composed of domain dictionaries to enhance characters’dependence.Second,domain rules were built to eliminate noise in entity relations and promote potential entity relation extraction.Finally,experiments were designed to verify the effectiveness of our proposed methods.Experimental results on two domains,including laser industry and unmanned ship,showed the superiority of our methods.The F1 value on laser industry entity,unmanned ship entity,laser industry relation,and unmanned ship relation datasets is improved by+1%,+6%,+2%,and+1%,respectively.In addition,the extraction accuracy of entity relation triplet reaches 83%and 76%on laser industry entity pair and unmanned ship entity pair datasets,respectively.

Targeted BERT Pre-training and Fine-Tuning Approach for Entity Relation Extraction

Entity relation extraction(ERE)is an important task in the field of information extraction.With the wide application of pre-training language model(PLM)in natural language processing(NLP),using PLM has become a brand new research direction of ERE.In this paper,BERT is used to extracting entityrelations,and a separated pipeline architecture is proposed.ERE was decomposed into entity-relation classification sub-task and entity-pair annotation sub-task.Both sub-tasks conduct the pre-training and fine-tuning independently.Combining dynamic and static masking,newVerb-MLM and Entity-MLM BERT pre-training tasks were put forward to enhance the correlation between BERT pre-training and TargetedNLPdownstream task-ERE.Inter-layer sharing attentionmechanismwas added to the model,sharing the attention parameters according to the similarity of the attention matrix.Contrast experiment on the SemEavl 2010 Task8 dataset demonstrates that the new MLM task and inter-layer sharing attention mechanism improve the performance of BERT on the entity relation extraction effectively.

Annotation and Joint Extraction of Scientific Entities and Relationships in NSFC Project Texts

Aiming at the lack of classification and good standard corpus in the task of joint entity and relationship extraction in the current Chinese academic field, this paper builds a dataset in management science that can be used for joint entity and relationship extraction, and establishes a deep learning model to extract entity and relationship information from scientific texts. With the definition of entity and relation classification, we build a Chinese scientific text corpus dataset based on the abstract texts of projects funded by the National Natural Science Foundation of China(NSFC) in 2018–2019. By combining the word2vec features with the clue word feature which is a kind of special style in scientific documents, we establish a joint entity relationship extraction model based on the Bi LSTM-CNN-CRF model for scientific information extraction. The dataset we constructed contains 13060 entities(not duplicated) and 9728 entity relation labels. In terms of entity prediction effect, the accuracy rate of the constructed model reaches 69.15%, the recall rate reaches 61.03%, and the F1 value reaches 64.83%. In terms of relationship prediction effect, the accuracy rate is higher than that of entity prediction, which reflects the effectiveness of the input mixed features and the integration of local features with CNN layer in the model.

Deep learning-based large-scale named entity recognition for anatomical region of mammalian brain

Background:Images of anatomical regions and neuron type distribution,as well as their related literature are valuable assets for neuroscience research.They are vital evidence and vehicles in discovering new phenomena and knowledge refinement through image and text big data.The knowledge acquired from image data generally echoes with the literature accumulated over the years.The knowledge within the literature can provide a comprehensive context for a deeper understanding of the image data.However,it is quite a challenge to manually identify the related literature and summarize the neuroscience knowledge in the large-scale corpus.Thus,neuroscientists are in dire need of an automated method to extract neuroscience knowledge from large-scale literature.Methods:A proposed deep learning model named BioBERT-CRF extracts brain region entities from the WhiteText dataset.This model takes advantage of BioBERT and CRF to predict entity labels while training.Results:The proposed deep learning model demonstrated comparable performance against or even outperforms the previous models on the WhiteText dataset.The BioBERT-CRF model has achieved the best average precision,recall,and F1 score of 81.3%,84.0%,and 82.6%,respectively.We used the BioBERT-CRF model to predict brain region entities in a large-scale PubMed abstract dataset and used a rule-based method to normalize all brain region entities to three neuroscience dictionaries.Conclusions:Our work shows that the BioBERT-CRF model can be well-suited for brain region entity extraction.The rankings of different brain region entities by their appearance in the large-scale corpus indicate the anatomical regions that researchers are most concerned about.

A Method for Extracting Electronic Medical Record Entities by Fusing Multichannel Self-Attention Mechanism with Location Relationship Features

With the implementation of the“Internet+”strategy,electronic medi-cal records are generally applied in the medicalfield.Deep mining of electronic medical record content data is an effective means to obtain medical knowledge and analyse patients’states,but the existing methods for extracting entities from electronic medical records have problems of redundant information,overlapping entities,and low accuracy rates.Therefore,this paper proposes an entity extrac-tion method for electronic medical records based on the network framework of BERT-BiLSTM,which incorporates a multichannel self-attention mechanism and location relationship features.First,the text input sequence was encoded using the BERT-BiLSTM network framework,and the global semantic information of the sentence was mined more deeply using the multichannel self-attention mech-anism.Then,the position relation characteristic was used to extract the local semantic message of the text,and the position relation characteristic of the word and the position embedding matrix of the whole sentence were obtained.Next,the extracted global semantic information was stitched with the positional embedding matrix of the sentence to obtain the current entity classification matrix.Finally,the proposed method was validated on the dataset of Chinese medical text entity relationship extraction and the 2010i2b2/VA relationship corpus,and the exper-imental results indicate that the proposed method surpasses existing methods in terms of precision,recall,F1 value and training time.

Data Set and Evaluation of Automated Construction of Financial Knowledge Graph

With the technological development of entity extraction, relationship extraction, knowledge reasoning, and entity linking, the research on knowledge graph has been carried out in full swing in recent years. To better promote the development of knowledge graph, especially in the Chinese language and in the financial industry, we built a high-quality data set, named financial research report knowledge graph(FR2 KG), and organized the automated construction of financial knowledge graph evaluation at the 2020 China Knowledge Graph and Semantic Computing Conference(CCKS2020). FR2 KG consists of 17,799 entities, 26,798 relationship triples, and 1,328 attribute triples covering 10 entity types, 19 relationship types, and 6 attributes. Participants are required to develop a constructor that will automatically construct a financial knowledge graph based on the FR2 KG. In addition, we summarized the technologies for automatically constructing knowledge graphs, and introduced the methods used by the winners and the results of this evaluation.

A New Multiword Expression Metric and Its Applications

Multiword Expressions （MWEs） appear frequently and ungrammatically in natural languages. Identifying MWEs in free texts is a very challenging problem. This paper proposes a knowledge-free, unsupervised, and languageindependent Multiword Expression Distance （MED）. The new metric is derived from an accepted physical principle, measures the distance from an n-gram to its semantics, and outperforms other state-of-the-art methods on MWEs in two applications： question answering and named entity extraction.