Network Configuration Entity Extraction Method Based on Transformer with Multi-Head Attention Mechanism

Nowadays,ensuring thequality of networkserviceshas become increasingly vital.Experts are turning toknowledge graph technology,with a significant emphasis on entity extraction in the identification of device configurations.This research paper presents a novel entity extraction method that leverages a combination of active learning and attention mechanisms.Initially,an improved active learning approach is employed to select the most valuable unlabeled samples,which are subsequently submitted for expert labeling.This approach successfully addresses the problems of isolated points and sample redundancy within the network configuration sample set.Then the labeled samples are utilized to train the model for network configuration entity extraction.Furthermore,the multi-head self-attention of the transformer model is enhanced by introducing the Adaptive Weighting method based on the Laplace mixture distribution.This enhancement enables the transformer model to dynamically adapt its focus to words in various positions,displaying exceptional adaptability to abnormal data and further elevating the accuracy of the proposed model.Through comparisons with Random Sampling(RANDOM),Maximum Normalized Log-Probability(MNLP),Least Confidence(LC),Token Entrop(TE),and Entropy Query by Bagging(EQB),the proposed method,Entropy Query by Bagging and Maximum Influence Active Learning(EQBMIAL),achieves comparable performance with only 40% of the samples on both datasets,while other algorithms require 50% of the samples.Furthermore,the entity extraction algorithm with the Adaptive Weighted Multi-head Attention mechanism(AW-MHA)is compared with BILSTM-CRF,Mutil_Attention-Bilstm-Crf,Deep_Neural_Model_NER and BERT_Transformer,achieving precision rates of 75.98% and 98.32% on the two datasets,respectively.Statistical tests demonstrate the statistical significance and effectiveness of the proposed algorithms in this paper.

SciCN:A Scientific Dataset for Chinese Named Entity Recognition

Named entity recognition(NER)is a fundamental task of information extraction(IE),and it has attracted considerable research attention in recent years.The abundant annotated English NER datasets have significantly promoted the NER research in the English field.By contrast,much fewer efforts are made to the Chinese NER research,especially in the scientific domain,due to the scarcity of Chinese NER datasets.To alleviate this problem,we present aChinese scientificNER dataset–SciCN,which contains entity annotations of titles and abstracts derived from 3,500 scientific papers.We manually annotate a total of 62,059 entities,and these entities are classified into six types.Compared to English scientific NER datasets,SciCN has a larger scale and is more diverse,for it not only contains more paper abstracts but these abstracts are derived from more research fields.To investigate the properties of SciCN and provide baselines for future research,we adapt a number of previous state-of-theart Chinese NER models to evaluate SciCN.Experimental results show that SciCN is more challenging than other Chinese NER datasets.In addition,previous studies have proven the effectiveness of using lexicons to enhance Chinese NER models.Motivated by this fact,we provide a scientific domain-specific lexicon.Validation results demonstrate that our lexicon delivers better performance gains than lexicons of other domains.We hope that the SciCN dataset and the lexicon will enable us to benchmark the NER task regarding the Chinese scientific domain and make progress for future research.The dataset and lexicon are available at:https://github.com/yangjingla/SciCN.git.

RoBGP:A Chinese Nested Biomedical Named Entity Recognition Model Based on RoBERTa and Global Pointer

Named Entity Recognition(NER)stands as a fundamental task within the field of biomedical text mining,aiming to extract specific types of entities such as genes,proteins,and diseases from complex biomedical texts and categorize them into predefined entity types.This process can provide basic support for the automatic construction of knowledge bases.In contrast to general texts,biomedical texts frequently contain numerous nested entities and local dependencies among these entities,presenting significant challenges to prevailing NER models.To address these issues,we propose a novel Chinese nested biomedical NER model based on RoBERTa and Global Pointer(RoBGP).Our model initially utilizes the RoBERTa-wwm-ext-large pretrained language model to dynamically generate word-level initial vectors.It then incorporates a Bidirectional Long Short-Term Memory network for capturing bidirectional semantic information,effectively addressing the issue of long-distance dependencies.Furthermore,the Global Pointer model is employed to comprehensively recognize all nested entities in the text.We conduct extensive experiments on the Chinese medical dataset CMeEE and the results demonstrate the superior performance of RoBGP over several baseline models.This research confirms the effectiveness of RoBGP in Chinese biomedical NER,providing reliable technical support for biomedical information extraction and knowledge base construction.

SHEL:a semantically enhanced hardware-friendly entity linking method

With the help of pre-trained language models,the accuracy of the entity linking task has made great strides in recent years.However,most models with excellent performance require fine-tuning on a large amount of training data using large pre-trained language models,which is a hardware threshold to accomplish this task.Some researchers have achieved competitive results with less training data through ingenious methods,such as utilizing information provided by the named entity recognition model.This paper presents a novel semantic-enhancement-based entity linking approach,named semantically enhanced hardware-friendly entity linking(SHEL),which is designed to be hardware friendly and efficient while maintaining good performance.Specifically,SHEL's semantic enhancement approach consists of three aspects:(1)semantic compression of entity descriptions using a text summarization model;(2)maximizing the capture of mention contexts using asymmetric heuristics;(3)calculating a fixed size mention representation through pooling operations.These series of semantic enhancement methods effectively improve the model's ability to capture semantic information while taking into account the hardware constraints,and significantly improve the model's convergence speed by more than 50%compared with the strong baseline model proposed in this paper.In terms of performance,SHEL is comparable to the previous method,with superior performance on six well-established datasets,even though SHEL is trained using a smaller pre-trained language model as the encoder.

A Federated Named Entity Recognition Model with Explicit Relation for Power Grid

The power grid operation process is complex,and many operation process data involve national security,business secrets,and user privacy.Meanwhile,labeled datasets may exist in many different operation platforms,but they cannot be directly shared since power grid data is highly privacysensitive.How to use these multi-source heterogeneous data as much as possible to build a power grid knowledge map under the premise of protecting privacy security has become an urgent problem in developing smart grid.Therefore,this paper proposes federated learning named entity recognition method for the power grid field,aiming to solve the problem of building a named entity recognition model covering the entire power grid process training by data with different security requirements.We decompose the named entity recognition(NER)model FLAT(Chinese NER Using Flat-Lattice Transformer)in each platform into a global part and a local part.The local part is used to capture the characteristics of the local data in each platform and is updated using locally labeled data.The global part is learned across different operation platforms to capture the shared NER knowledge.Its local gradients fromdifferent platforms are aggregated to update the global model,which is further delivered to each platform to update their global part.Experiments on two publicly available Chinese datasets and one power grid dataset validate the effectiveness of our method.

A Two-Phase Paradigm for Joint Entity-Relation Extraction

An exhaustive study has been conducted to investigate span-based models for the joint entity and relation extraction task.However,these models sample a large number of negative entities and negative relations during the model training,which are essential but result in grossly imbalanced data distributions and in turn cause suboptimal model performance.In order to address the above issues,we propose a two-phase paradigm for the span-based joint entity and relation extraction,which involves classifying the entities and relations in the first phase,and predicting the types of these entities and relations in the second phase.The two-phase paradigm enables our model to significantly reduce the data distribution gap,including the gap between negative entities and other entities,aswell as the gap between negative relations and other relations.In addition,we make the first attempt at combining entity type and entity distance as global features,which has proven effective,especially for the relation extraction.Experimental results on several datasets demonstrate that the span-based joint extraction model augmented with the two-phase paradigm and the global features consistently outperforms previous state-ofthe-art span-based models for the joint extraction task,establishing a new standard benchmark.Qualitative and quantitative analyses further validate the effectiveness the proposed paradigm and the global features.

Corpus of Carbonate Platforms with Lexical Annotations for Named Entity Recognition

An obviously challenging problem in named entity recognition is the construction of the kind data set of entities.Although some research has been conducted on entity database construction,the majority of them are directed at Wikipedia or the minority at structured entities such as people,locations and organizational nouns in the news.This paper focuses on the identification of scientific entities in carbonate platforms in English literature,using the example of carbonate platforms in sedimentology.Firstly,based on the fact that the reasons for writing literature in key disciplines are likely to be provided by multidisciplinary experts,this paper designs a literature content extraction method that allows dealing with complex text structures.Secondly,based on the literature extraction content,we formalize the entity extraction task(lexicon and lexical-based entity extraction)for entity extraction.Furthermore,for testing the accuracy of entity extraction,three currently popular recognition methods are chosen to perform entity detection in this paper.Experiments show that the entity data set provided by the lexicon and lexical-based entity extraction method is of significant assistance for the named entity recognition task.This study presents a pilot study of entity extraction,which involves the use of a complex structure and specialized literature on carbonate platforms in English.

The Entity Relationship Extraction Method Using Improved RoBERTa and Multi-Task Learning

There is a growing amount of data uploaded to the internet every day and it is important to understand the volume of those data to find a better scheme to process them.However,the volume of internet data is beyond the processing capabilities of the current internet infrastructure.Therefore,engineering works using technology to organize and analyze information and extract useful information are interesting in both industry and academia.The goal of this paper is to explore the entity relationship based on deep learning,introduce semantic knowledge by using the prepared language model,develop an advanced entity relationship information extraction method by combining Robustly Optimized BERT Approach(RoBERTa)and multi-task learning,and combine the intelligent characters in the field of linguistic,called Robustly Optimized BERT Approach+Multi-Task Learning(RoBERTa+MTL).To improve the effectiveness of model interaction,multi-task teaching is used to implement the observation information of auxiliary tasks.Experimental results show that our method has achieved an accuracy of 88.95 entity relationship extraction,and a further it has achieved 86.35%of accuracy after being combined with multi-task learning.

Entity generation algorithm based on reference expansion

The extraction and understanding of text knowledge become increasingly crucial in the age of big data.One of the current research areas in the field of natural language processing(NLP)is how to accurately understand the text and collect accurate linguistic information because Chinese vocabulary is diverse and ambiguous.This paper mainly studies the candidate entity generation module of the entity link system.The candidate entity generation module constructs an entity reference expansion algorithm to improve the recall rate of candidate entities.In order to improve the efficiency of the connection algorithm of the entire system while ensuring the recall rate of candidate entities,we design a graph model filtering algorithm that fuses shallow semantic information to filter the list of candidate entities,and verify and analyze the efficiency of the algorithm through experiments.By analyzing the related technology of the entity linking algorithm,we study the related technology of candidate entity generation and entity disambiguation,improve the traditional entity linking algorithm,and give an innovative and practical entity linking model.The recall rate exceeds 82%,and the link accuracy rate exceeds 73%.Efficient and accurate entity linking can help machines to better understand text semantics,further promoting the development of NLP and improving the users’knowledge acquisition experience on the text.

Chinese Cyber Threat Intelligence Named Entity Recognition via RoBERTa-wwm-RDCNN-CRF

In recent years,cyber attacks have been intensifying and causing great harm to individuals,companies,and countries.The mining of cyber threat intelligence(CTI)can facilitate intelligence integration and serve well in combating cyber attacks.Named Entity Recognition(NER),as a crucial component of text mining,can structure complex CTI text and aid cybersecurity professionals in effectively countering threats.However,current CTI NER research has mainly focused on studying English CTI.In the limited studies conducted on Chinese text,existing models have shown poor performance.To fully utilize the power of Chinese pre-trained language models(PLMs)and conquer the problem of lengthy infrequent English words mixing in the Chinese CTIs,we propose a residual dilated convolutional neural network(RDCNN)with a conditional random field(CRF)based on a robustly optimized bidirectional encoder representation from transformers pre-training approach with whole word masking(RoBERTa-wwm),abbreviated as RoBERTa-wwm-RDCNN-CRF.We are the first to experiment on the relevant open source dataset and achieve an F1-score of 82.35%,which exceeds the common baseline model bidirectional encoder representation from transformers(BERT)-bidirectional long short-term memory(BiLSTM)-CRF in this field by about 19.52%and exceeds the current state-of-the-art model,BERT-RDCNN-CRF,by about 3.53%.In addition,we conducted an ablation study on the encoder part of the model to verify the effectiveness of the proposed model and an in-depth investigation of the PLMs and encoder part of the model to verify the effectiveness of the proposed model.The RoBERTa-wwm-RDCNN-CRF model,the shared pre-processing,and augmentation methods can serve the subsequent fundamental tasks such as cybersecurity information extraction and knowledge graph construction,contributing to important applications in downstream tasks such as intrusion detection and advanced persistent threat(APT)attack detection.

A Weakly-Supervised Method for Named Entity Recognition of Agricultural Knowledge Graph

It is significant for agricultural intelligent knowledge services using knowledge graph technology to integrate multi-source heterogeneous crop and pest data and fully mine the knowledge hidden in the text.However,only some labeled data for agricultural knowledge graph domain training are available.Furthermore,labeling is costly due to the need for more data openness and standardization.This paper proposes a novel model using knowledge distillation for a weakly supervised entity recognition in ontology construction.Knowledge distillation between the target and source data domain is performed,where Bi-LSTM and CRF models are constructed for entity recognition.The experimental result is shown that we only need to label less than one-tenth of the data for model training.Furthermore,the agricultural domain ontology is constructed by BILSTM-CRF named entity recognition model and relationship extraction model.Moreover,there are a total of 13,983 entities and 26,498 relationships built in the neo4j graph database.

Dart Games Optimizer with Deep Learning-Based Computational Linguistics Named Entity Recognition

Computational linguistics is an engineering-based scientific discipline.It deals with understanding written and spoken language from a computational viewpoint.Further,the domain also helps construct the artefacts that are useful in processing and producing a language either in bulk or in a dialogue setting.Named Entity Recognition(NER)is a fundamental task in the data extraction process.It concentrates on identifying and labelling the atomic components from several texts grouped under different entities,such as organizations,people,places,and times.Further,the NER mechanism identifies and removes more types of entities as per the requirements.The significance of the NER mechanism has been well-established in Natural Language Processing(NLP)tasks,and various research investigations have been conducted to develop novel NER methods.The conventional ways of managing the tasks range from rule-related and hand-crafted feature-related Machine Learning(ML)techniques to Deep Learning(DL)techniques.In this aspect,the current study introduces a novel Dart Games Optimizer with Hybrid Deep Learning-Driven Computational Linguistics(DGOHDL-CL)model for NER.The presented DGOHDL-CL technique aims to determine and label the atomic components from several texts as a collection of the named entities.In the presented DGOHDL-CL technique,the word embed-ding process is executed at the initial stage with the help of the word2vec model.For the NER mechanism,the Convolutional Gated Recurrent Unit(CGRU)model is employed in this work.At last,the DGO technique is used as a hyperparameter tuning strategy for the CGRU algorithm to boost the NER’s outcomes.No earlier studies integrated the DGO mechanism with the CGRU model for NER.To exhibit the superiority of the proposed DGOHDL-CL technique,a widespread simulation analysis was executed on two datasets,CoNLL-2003 and OntoNotes 5.0.The experimental outcomes establish the promising performance of the DGOHDL-CL technique over other models.

Moderate scale and realization potential of new citrus-planting business entities in hilly and mountainous areas in China

The natural and economic conditions of agricultural resources vary greatly in China,especially in hilly and mountainous areas.The phenomenon of land fragmentation has become increasingly prominent,so that large-scale operations cannot be fully achieved in a short period of time,and the centralized and continuous scale of operations cannot be realized in China.In addition,with increasingly strict farmland protection and agricultural land use control systems,the issues of nongrain production and nonagricultural cultivated land use have become increasingly restricted.Thus,it is quite necessary to determine the appropriate scale of operations and the means to achieve moderately scaled operations for the new business entities.On the basis of microsurvey data for 108 new citrus-planting business entities in the modern agricultural park in the Chongqing's Jiangjin District,an area with long citrus-planting history,we measured the moderate scale of new citrus-planting business entities to maximize profit using a translog growth model.According to the projection pursuit model,we evaluated the suitability conditions of citrus planting in garden land,woodland,grassland,and general cultivated land in the study area.We then explored the potential for new moderate-scale business entities within different farming radii.The results showed that large-scale planting conditions of citrus in garden land,forest land,grassland,and general cultivated land in the study area were suitable,and the proportion of high-suitable and mediumsuitable land was 73.42%.Under the existing social and economic conditions,the moderate scale of new citrus-planting business entities in the study area was 1.8–2.7 hm^(2).In particular,its rankings from large to small were agricultural enterprises(17.19–25.78 hm^(2)),farmer cooperatives(16.88–25.33 hm^(2)),big growers and breeders(6.39–9.59 hm^(2)),and family farms(5.02–7.53 hm^(2)).In the sample of 108 households,only 47.22%of the entities achieved moderate-scale operation,of which 25%of the entities achieved a scale greater than moderate operation.However,52.78%of the entities achieved a scale of less than moderate operation.These entities would have to transfer the surrounding adjacent garden land,forest land,grassland,or general cultivated land to achieve largescale land management.The land area of the adjacent gardens in a 1000 m farming radius could meet the moderate-scale operation demand of the vast majority of new business entities.For the vast majority of new business entities,the land area of the adjacent woodland,grassland,and general cultivated land could supplement the garden land to achieve moderate-scale operation in a 500 m farming radius.If the land area with moderate suitability and high suitability is prioritized,the land area in the adjacent gardens in the 1000 m farming radius could meet the moderate-scale operation demand for the vast majority of new business entities.Within the 500 m farming radius,the vast majority of new business entities have achieved moderate-scale operations if the land area adjacent to forest land,grassland,and general arable land was supplemented by gardens;however,a few entities could not achieve moderate-scale operations.From the village perspective,gardens were prioritized.Sixteen villages had planting areas and planting suitability that exceeded the average level of the study area,accounting for 23.53%.If combined with the reserve potential of the garden land,eight villages could improve their potential,accounting for 11.76%.Therefore,the conditions of large-scale citrus planting in the study area should be further improved,and the scale expansion of new citrus-planting business entities should receive additional scientific guidance.

Data Masking for Chinese Electronic Medical Records with Named Entity Recognition

With the rapid development of information technology,the electronifi-cation of medical records has gradually become a trend.In China,the population base is huge and the supporting medical institutions are numerous,so this reality drives the conversion of paper medical records to electronic medical records.Electronic medical records are the basis for establishing a smart hospital and an important guarantee for achieving medical intelligence,and the massive amount of electronic medical record data is also an important data set for conducting research in the medical field.However,electronic medical records contain a large amount of private patient information,which must be desensitized before they are used as open resources.Therefore,to solve the above problems,data masking for Chinese electronic medical records with named entity recognition is proposed in this paper.Firstly,the text is vectorized to satisfy the required format of the model input.Secondly,since the input sentences may have a long or short length and the relationship between sentences in context is not negligible.To this end,a neural network model for named entity recognition based on bidirectional long short-term memory(BiLSTM)with conditional random fields(CRF)is constructed.Finally,the data masking operation is performed based on the named entity recog-nition results,mainly using regular expression filtering encryption and principal component analysis(PCA)word vector compression and replacement.In addi-tion,comparison experiments with the hidden markov model(HMM)model,LSTM-CRF model,and BiLSTM model are conducted in this paper.The experi-mental results show that the method used in this paper achieves 92.72%Accuracy,92.30%Recall,and 92.51%F1_score,which has higher accuracy compared with other models.

沉积模拟实验信息系统的设计与实现

多年来,国内外地质学家在沉积模拟实验研究中积累了海量的实验数据和资料,但目前缺少对此类信息进行规范化管理和共享应用的软件系统。对此,提出浏览器/服务器(B/S)模式的沉积模拟实验信息系统,实现了沉积模拟实验的实验设计方案、实验观测数据、实验结果等信息的集成管理、高效检索和可视化分析等功能,从而为沉积模拟实验的网上信息化管理与分析应用提供可行方案。最后,以某油田辫状河储层沉积模拟实验为例,介绍了系统中实验项目检索、实验详情浏览、砂体测量数据的统计分析等功能的应用效果,说明了本系统的实用性。本系统的研发与应用可极大地方便沉积模拟实验数据的管理、分享及分析工作。

Optimization of Rural Governance Structure under the Development of New Agricultural Management Entities

The positive impact of the development of new agricultural business entities and their institutional systems on the optimization of rural governance structures can be examined from various perspectives,including the state,market,rural society,urban-rural relations,and grassroots governance structure in rural communities.The development of these new agricultural business entities has not only redefined but also restructured the power distribution and governance patterns in rural developing countries,markets,and villages.The rural social order has evolved into a ternary mutual structure governance pattern,often referred to as the“state market rural”model.This transformation has prompted adjustments in the national economic and social policy structure and management systems at both macro and micro levels.It has led to the reshaping of power dynamics,benefit distribution,and governance structures in both urban and rural areas,resulting in significant changes to the economic and social fabric of rural regions.Furthermore,the grassroots governance structure in rural society,characterized by“township governance and village governance,”is undergoing continuous development and improvement.This transition is marked by a shift towards a collaborative governance structure that encourages diverse participation.Building upon the aforementioned optimizations,the rural governance structure now exhibits new characteristics.These include a more extensive and diverse range of rural governance mechanisms,increased openness in governance processes,and a heightened synergy among various governance mechanisms.This dynamic evolution reflects a richer,more diverse,and more open approach to rural governance,fostering a stronger collaborative effort in the pursuit of effective governance.

医学领域知识融合研究进展

医学领域知识融合旨在将分散在各个知识图谱或不同数据源中的医学知识进行整合,形成一个更全面的知识图谱,在提高知识质量、扩大规模、提高医学知识利用率和共享性等方面具有促进作用。围绕知识融合的问题和解决方案,首先系统地梳理了医学领域知识融合的定义、评价指标及数据集;分类讨论了知识融合过程中存在的问题与挑战;然后从问题、技术两个维度,综述了目前知识融合中实体对齐、实体链接任务各方法的优势与不足;详细讨论和总结了医学领域知识融合每一类问题的相关解决方案;最后,总结并展望了医学领域知识融合的发展方向。

新型农业经营主体对劳动力流动的影响效应

新型农业经营主体是推进农业农村现代化和乡村振兴的有机载体,有效推动了农业规模化经营,有助于引导和促进劳动力流动,从而实现农村劳动力的优化配置。基于中国社会科学院农村发展研究所中国乡村振兴调查(CRRS)数据,选用Logit模型研究新型农业经营主体对劳动力流动的影响效应。研究表明:第一,新型农业经营主体会促进劳动力的回流,但对劳动力流出无显著影响;第二,有新型农业经营主体的村庄更容易吸引年龄较低和受教育程度较高的农村劳动力的回流;第三,在新型农业经营主体吸引劳动力回流的过程中,参与过非农职业技术培训的农村劳动力更容易回流。基于此,从管理与服务方式、培育方向、人才培养和体制机制四个角度提出了针对性的政策建议,为引导和优化农村劳动力的合理配置提供借鉴经验,对于当前农村经济社会可持续发展、推动建设农业强国具有重要的现实价值。

建设现代化产业体系:理论基础、演进逻辑与实践路径——基于实体经济支撑视角

现代化产业体系是实现高质量发展的重要承载体,必须把现代化产业体系建设的支撑点放在实体经济上。建设以实体经济为支撑的现代化产业体系要遵循“单点应用、局部优化、体系融合、生态重构”四个层面的演进逻辑,即解决关键核心技术的“卡脖子”问题,实现多业务环节和流程系统的局部集成优化,实现现代服务业与现代产业加速融合,实现数字经济和实体经济深度融合。从产业安全提升、产业结构调整、产业跨界融合、产业要素流动等的价值维度认识到现代化产业体系建设的关键问题,应聚焦融合实践路径,推动三次产业融合发展;聚焦循环实践路径,提升全要素生产率;聚焦数字化实践路径,巩固优势传统产业领先地位;聚焦生态化实践路径,加快实现高水平科技自立自强。

认罪认罚从宽制度的再完善:以轻微犯罪治理为场域

目前,我国已经进入轻微犯罪时代。通过认罪认罚从宽制度治理轻微犯罪是重要的刑事司法途径。然而,轻微犯罪治理与认罪认罚从宽制度之间仍存在一定的不适性,特别是实体与程序的衔接不畅、治理需求与制度供给的不一致等。因此,应当强化轻微犯罪治理与认罪认罚从宽制度之间的契合度与协同性,以轻微犯罪治理的实际需求为导向,按照刑事一体化的理念,不断完善认罪认罚从宽制度及其相关的实施机制。当前,刑法与认罪认罚从宽制度之间的制度性“沟壑”亟待填补。为此,既需要调试现行刑法理论的互斥部分,也要重新审视刑法总则相关规定的适宜性和有效性,进而及时启动必要的实体性立法修正程序。《刑事诉讼法》修改在即,整合性的程序回应也势在必行,独立的认罪认罚从宽协商程序、更加健全的量刑协商机制等均是轻罪治理的主要关切点。