RoBGP:A Chinese Nested Biomedical Named Entity Recognition Model Based on RoBERTa and Global Pointer

Named Entity Recognition(NER)stands as a fundamental task within the field of biomedical text mining,aiming to extract specific types of entities such as genes,proteins,and diseases from complex biomedical texts and categorize them into predefined entity types.This process can provide basic support for the automatic construction of knowledge bases.In contrast to general texts,biomedical texts frequently contain numerous nested entities and local dependencies among these entities,presenting significant challenges to prevailing NER models.To address these issues,we propose a novel Chinese nested biomedical NER model based on RoBERTa and Global Pointer(RoBGP).Our model initially utilizes the RoBERTa-wwm-ext-large pretrained language model to dynamically generate word-level initial vectors.It then incorporates a Bidirectional Long Short-Term Memory network for capturing bidirectional semantic information,effectively addressing the issue of long-distance dependencies.Furthermore,the Global Pointer model is employed to comprehensively recognize all nested entities in the text.We conduct extensive experiments on the Chinese medical dataset CMeEE and the results demonstrate the superior performance of RoBGP over several baseline models.This research confirms the effectiveness of RoBGP in Chinese biomedical NER,providing reliable technical support for biomedical information extraction and knowledge base construction.

GeoNER:Geological Named Entity Recognition with Enriched Domain Pre-Training Model and Adversarial Training

As important geological data,a geological report contains rich expert and geological knowledge,but the challenge facing current research into geological knowledge extraction and mining is how to render accurate understanding of geological reports guided by domain knowledge.While generic named entity recognition models/tools can be utilized for the processing of geoscience reports/documents,their effectiveness is hampered by a dearth of domain-specific knowledge,which in turn leads to a pronounced decline in recognition accuracy.This study summarizes six types of typical geological entities,with reference to the ontological system of geological domains and builds a high quality corpus for the task of geological named entity recognition(GNER).In addition,Geo Wo BERT-adv BGP(Geological Word-base BERTadversarial training Bi-directional Long Short-Term Memory Global Pointer)is proposed to address the issues of ambiguity,diversity and nested entities for the geological entities.The model first uses the fine-tuned word granularitybased pre-training model Geo Wo BERT(Geological Word-base BERT)and combines the text features that are extracted using the Bi LSTM(Bi-directional Long Short-Term Memory),followed by an adversarial training algorithm to improve the robustness of the model and enhance its resistance to interference,the decoding finally being performed using a global association pointer algorithm.The experimental results show that the proposed model for the constructed dataset achieves high performance and is capable of mining the rich geological information.

A Novel Optimization Scheme for Named Entity Recognition with Pre-trained Language Models

Named Entity Recognition(NER)is crucial for extracting structured information from text.While traditional methods rely on rules,Conditional Random Fields(CRFs),or deep learning,the advent of large-scale Pre-trained Language Models(PLMs)offers new possibilities.PLMs excel at contextual learning,potentially simplifying many natural language processing tasks.However,their application to NER remains underexplored.This paper investigates leveraging the GPT-3 PLM for NER without fine-tuning.We propose a novel scheme that utilizes carefully crafted templates and context examples selected based on semantic similarity.Our experimental results demonstrate the feasibility of this approach,suggesting a promising direction for harnessing PLMs in NER.

Three-Dimensional Tidal Model and Its Application to Numerical Simulation of Water Quality in Coastal Waters

The turbulence mechanism plays an important part in the mixing process and momentum transfer of turbulence. A three-dimensional Prandtl mixing length tidal model has been developed to simulate tidal flows and water quality. The eddy viscosities and diffusivities are computed from the Prandtl mixing length model. In order to model the water quality of an estuary or coastal area many interdependent processes need to be simulated. These may be conveniently separated into three main groups: transport and mixing processes, biochemical interaction of water quality variables and the utilization and re-cycling of nutrients by living matter. The model simulates full oxygen and nutrient balance, primary productivity and the transport, reaction mechanism and fate of pollutants over tidal time-scales. The model is applied to numerical simulation of tidal flows and water quality in Dalian Bay. The model has been calibrated against a limited data set of historical water quality observations and in general demonstrates excellent agreement with all available data.

The three-dimension model for the rock-breaking mechanism of disc cutter andanalysis of rock-breaking forces

To study the rock deformation with three- dimensional model under rolling forces of disc cutter, by car- rying out the circular-grooving test with disc cutter rolling around on the rock, the rock mechanical behavior under rolling disc cutter is studied, the mechanical model of disc cutter rolling around the groove is established, and the the- ory of single-point and double-angle variables is proposed. Based on this theory, the physics equations and geometric equations of rock mechanical behavior under disc cutters of tunnel boring machine （TBM） are studied, and then the bal- ance equations of interactive forces between disc cutter and rock are established. Accordingly, formulas about normal force, rolling force and side force of a disc cutter are de- rived, and their validity is studied by tests. Therefore, a new method and theory is proposed to study rock- breaking mech- anism of disc cutters.

SHEL:a semantically enhanced hardware-friendly entity linking method

With the help of pre-trained language models,the accuracy of the entity linking task has made great strides in recent years.However,most models with excellent performance require fine-tuning on a large amount of training data using large pre-trained language models,which is a hardware threshold to accomplish this task.Some researchers have achieved competitive results with less training data through ingenious methods,such as utilizing information provided by the named entity recognition model.This paper presents a novel semantic-enhancement-based entity linking approach,named semantically enhanced hardware-friendly entity linking(SHEL),which is designed to be hardware friendly and efficient while maintaining good performance.Specifically,SHEL's semantic enhancement approach consists of three aspects:(1)semantic compression of entity descriptions using a text summarization model;(2)maximizing the capture of mention contexts using asymmetric heuristics;(3)calculating a fixed size mention representation through pooling operations.These series of semantic enhancement methods effectively improve the model's ability to capture semantic information while taking into account the hardware constraints,and significantly improve the model's convergence speed by more than 50%compared with the strong baseline model proposed in this paper.In terms of performance,SHEL is comparable to the previous method,with superior performance on six well-established datasets,even though SHEL is trained using a smaller pre-trained language model as the encoder.

Three-dimensional frequency-domain full waveform inversion based on the nearly-analytic discrete method

The nearly analytic discrete(NAD)method is a kind of finite difference method with advantages of high accuracy and stability.Previous studies have investigated the NAD method for simulating wave propagation in the time-domain.This study applies the NAD method to solving three-dimensional(3D)acoustic wave equations in the frequency-domain.This forward modeling approach is then used as the“engine”for implementing 3D frequency-domain full waveform inversion(FWI).In the numerical modeling experiments,synthetic examples are first given to show the superiority of the NAD method in forward modeling compared with traditional finite difference methods.Synthetic 3D frequency-domain FWI experiments are then carried out to examine the effectiveness of the proposed methods.The inversion results show that the NAD method is more suitable than traditional methods,in terms of computational cost and stability,for 3D frequency-domain FWI,and represents an effective approach for inversion of subsurface model structures.

Chinese Named Entity Recognition with Character-Level BLSTM and Soft Attention Model

Unlike named entity recognition(NER)for English,the absence of word boundaries reduces the final accuracy for Chinese NER.To avoid accumulated error introduced by word segmentation,a deep model extracting character-level features is carefully built and becomes a basis for a new Chinese NER method,which is proposed in this paper.This method converts the raw text to a character vector sequence,extracts global text features with a bidirectional long short-term memory and extracts local text features with a soft attention model.A linear chain conditional random field is also used to label all the characters with the help of the global and local text features.Experiments based on the Microsoft Research Asia(MSRA)dataset are designed and implemented.Results show that the proposed method has good performance compared to other methods,which proves that the global and local text features extracted have a positive influence on Chinese NER.For more variety in the test domains,a resume dataset from Sina Finance is also used to prove the effectiveness of the proposed method.

Tracking topological entity changes in 3D collaborative modeling systems

One of the key problems in collaborative geometric modeling systems is topological entity correspondence when topolog- ical structure of geometry models on collaborative sites changes, ha this article, we propose a solution for tracking topological entity alterations in 3D collaborative modeling environment. We firstly make a thorough analysis and detailed categorization on the altera- tion properties and causations for each type of topological entity, namely topological face and topological edge. Based on collabora- tive topological entity naming mechanism, a data structure called TEST （Topological Entity Structure Tree） is introduced to track the changing history and current state of each topological entity, to embody the relationship among topological entities. Rules and algo- rithms are presented for identification of topological entities referenced by operations for correct execution and model consistency. The algorithm has been verified within the prototype we have implemented with ACIS.

Entity generation algorithm based on reference expansion

The extraction and understanding of text knowledge become increasingly crucial in the age of big data.One of the current research areas in the field of natural language processing(NLP)is how to accurately understand the text and collect accurate linguistic information because Chinese vocabulary is diverse and ambiguous.This paper mainly studies the candidate entity generation module of the entity link system.The candidate entity generation module constructs an entity reference expansion algorithm to improve the recall rate of candidate entities.In order to improve the efficiency of the connection algorithm of the entire system while ensuring the recall rate of candidate entities,we design a graph model filtering algorithm that fuses shallow semantic information to filter the list of candidate entities,and verify and analyze the efficiency of the algorithm through experiments.By analyzing the related technology of the entity linking algorithm,we study the related technology of candidate entity generation and entity disambiguation,improve the traditional entity linking algorithm,and give an innovative and practical entity linking model.The recall rate exceeds 82%,and the link accuracy rate exceeds 73%.Efficient and accurate entity linking can help machines to better understand text semantics,further promoting the development of NLP and improving the users’knowledge acquisition experience on the text.

Coding of Topological Entities in Feature-Based Parametric Modeling System

How to identify topological entities during rebuilding features is a critical problem in Feature-Based Parametric Modeling System (FBPMS). In the article, authors proposes a new coding approach to distinguish different entities. The coding mechanism is expatiated,and some typical examples are presented. At last, the algorithm of decoding is put forward based on set theory.

自适应特征融合的多模态实体对齐研究

多模态数据间交互式任务的兴起对于综合利用不同模态的知识提出了更高的要求,因此融合不同模态知识的多模态知识图谱应运而生.然而,现有多模态知识图谱存在图谱知识不完整的问题,严重阻碍对信息的有效利用.缓解此问题的有效方法是通过实体对齐进行知识图谱补全.当前多模态实体对齐方法以固定权重融合多种模态信息,在融合过程中忽略不同模态信息贡献的差异性.为解决上述问题,设计一套自适应特征融合机制,根据不同模态数据质量动态融合实体结构信息和视觉信息.此外,考虑到视觉信息质量不高、知识图谱之间的结构差异也影响实体对齐的效果,本文分别设计提升视觉信息有效利用率的视觉特征处理模块以及缓和结构差异性的三元组筛选模块.在多模态实体对齐任务上的实验结果表明,提出的多模态实体对齐方法的性能优于当前最好的方法.

基于RoBERTa-Span-Attack的标签指针网络军事命名实体识别

军事领域文本中存在大量军事实体信息,准确识别这些信息是军事文本信息提取和构建军事知识图谱的基础性任务。首先,提出了一种基于RoBERTa预训练模型、跨度和对抗训练的标签指针网络的融合深度模型(RoBERTa-Span-Attack),用于中文军事命名实体识别;然后,采用了一种基于Span的标签指针网络,同时完成实体的起止位置和类别的识别任务;最后,在模型训练过程中加入对抗训练策略,通过添加一些扰动来生成对抗样本进行训练。在军事领域数据集上的实验结果表明:所提出的军事领域命名实体识别模型相较于BERT-CRF、BERT-Softmax和BERT-Span,在识别准确度上具有更优的效果。

面向技术识别的专利实体抽取--以类脑智能领域为例

[研究目的]专利实体抽取是基于专利文本的技术识别的基础。目前专利实体抽取任务面临自动化程度和准确率较低等问题,该研究从两方面对此进行改进:一是建立特定领域的高质量专利语料库,二是将先进的算法模型运用到专利实体抽取中。[研究方法]定义了包含13种实体类型的细粒度信息体系,并据此对921篇类脑智能专利的标题和摘要进行人工标注,此后运用Bert-BiLSTM-CRF模型,融合深度学习和机器学习对类脑智能专利实体进行识别。[研究结论]模型在总体上获得0.8的准确率、召回率和F1值,不同类型实体的识别效果具有差异。为了验证模型的性能,设计了几个对比实验。结果显示,微调数据和增加训练规模可以提高模型性能,本模型性能优于同时期一些经典模型。

汽车故障知识图谱构建及应用研究

知识图谱技术对汽车高效的故障诊断具有重要的意义,现有汽车故障知识图谱构建存在着实体识别模型效果不佳、无法解决嵌套实体等问题。针对上述问题,通过采用全词掩码的预训练语义模型、加入对抗训练和改进嵌套实体识别模型的方式提高实体识别模型效果,提出了一种改进的嵌套实体识别模型。实验结果表明,所提模型F1值(F_(1))、精确率(P)和召回率(R)相比基线模型分别提高了3.56%、4.08%、3.05%,相比其他模型也有不同程度的提高,验证了所提模型对汽车维修领域实体识别具有显著效果。同时,基于构建的汽车故障知识图谱,实现了汽车故障知识智能问答原型系统,展示了知识图谱技术在汽车故障诊断与维护领域的应用前景。

基于连续提示注入与指针网络的农业病害命名实体识别

针对农业病害领域命名实体识别过程中存在的预训练语言模型利用不充分、外部知识注入利用率低、嵌套命名实体识别率低的问题,本文提出基于连续提示注入和指针网络的命名实体识别模型CP-MRC(Continuous prompts for machine reading comprehension)。该模型引入BERT(Bidirectional encoder representation from transformers)预训练模型,通过冻结BERT模型原有参数,保留其在预训练阶段获取到的文本表征能力;为了增强模型对领域数据的适用性,在每层Transformer中插入连续可训练提示向量;为提高嵌套命名实体识别的准确性,采用指针网络抽取实体序列。在自建农业病害数据集上开展了对比实验,该数据集包含2933条文本语料,8个实体类型,共10414个实体。实验结果显示,CP-MRC模型的精确率、召回率、F1值达到83.55%、81.4%、82.4%,优于其他模型;在病原、作物两类嵌套实体的识别率较其他模型F1值提升3个百分点和13个百分点,嵌套实体识别率明显提升。本文提出的模型仅采用少量可训练参数仍然具备良好识别性能,为较大规模预训练模型在信息抽取任务上的应用提供了思路。

基于知识图谱的建筑施工安全风险量化与分析

为了提高建筑施工安全风险管理的信息化水平,以建筑施工活动及事故风险类型为研究对象,建立施工安全知识图谱。通过知识图谱改进作业条件危险性评价法(LEC)实现安全风险的定量计算,并基于知识图谱进行风险位置识别和不安全因素分析。研究提出安全风险虚体实化理念,实现了安全风险信息在数字空间实体化表达;基于建筑信息模型(Building Information Modeling, BIM)和知识图谱技术,建立了建筑施工安全风险信息模型(Building Construction Safety Risk Information Model, BCSRIM)。该模型有效避免了传统LEC法中主观因素产生的影响,实现了建筑施工安全风险定量计算、风险位置识别、风险分析及可视化管理。利用Revit二次开发技术,在Microsoft Visual Studio中使用C#语言连接Neo4j图数据库,完成了基于知识图谱的BCSRIM的开发。试验显示,研究提出的BCSRIM对提高施工现场的管理水平具有较高的实用价值。

新型农业经营主体促进山区小农户机械化的“滴涓效应”研究——以云南山区为例

本文旨在研究促进山区小农户与现代农业衔接的路径和机理。通过对云南省山区小农户的调查数据,采用回归和调节模型进行实证分析,验证促进山区小农户机械化水平的外部和内部路径以及相关机理的有效性,并探讨新型农业主体对山区小农户机械化水平影响的“涓滴效应”。研究结果表明,山区小农户的机械化水平主要受其社会和自然资源的影响以及内部互助行为的作用。新型农业经营主体在提升山区小农户整体机械化水平方面并不显著,但在提升部分山区小农户的机械化水平方面具有显著作用。农业企业的带动效应对于提升“非合作社社员”和“未获得免费培训”的山区小农户机械化水平起到明显的促进作用。而农民合作社仅在提升合作社社员的机械化水平方面发挥显著作用。因此,建议加大山地农业机械的研发投入,增强新型农业经营主体在提升小农户整体现代农业水平方面的作用,以促进小农户与现代农业的有机衔接。

因果关系表示增强的跨领域命名实体识别

跨领域命名实体识别在现实应用中,尤其在目标领域数据稀缺的小样本场景中具有重要价值。然而,现有方法主要是通过特征表示或模型参数共享实现的跨领域实体能力迁移,未充分考虑由于样本选择偏差而引起的虚假相关性问题。为了解决跨领域中的虚假相关性问题,提出一种因果关系表示增强的跨领域命名实体识别模型,将源域的语义特征表示与目标域的语义特征表示进行融合,生成一种增强的上下文语义特征表示。通过结构因果模型捕捉增强后的特征变量与标签之间的因果关系。在目标域中应用因果干预和反事实推断策略,提取存在的直接因果效应,从而进一步缓解特征与标签之间的虚假相关性问题。该方法在公共数据集上进行了实验,实验结果得到了显著提高。

社交媒体数据中水灾事件求助信息提取模型

由于社交媒体平台上所发布的非结构化信息存在数据不一致、重要程度不同等问题,使自动准确抽取所需信息并标注受灾级别成为一个有挑战性的工作。因此,结合形式概念分析(FCA)、词共现关系和上下文语义信息构建了水灾事件知识体系。利用所构建的知识体系,基于TencentPretrain框架对大规模语言预训练模型(LLM)进行指令微调,构建了ChatFlowFlood信息抽取模型,可以在少量人工标记情况下,准确自动抽取被困情况、紧缺物资等信息;在信息抽取模型的基础上,通过模糊层次分析法(FAHP)和CRITIC法(CRiteria Importance Through Intercriteria Correlation)主客观结合评定求助信息的救援优先级,帮助决策者理解灾情紧急程度。实验结果表明,在中文社交媒体数据上,与ChatFlow-7B模型相比,ChatFlowFlood模型的FBERT指标提升了73.09%。