GeoNER:Geological Named Entity Recognition with Enriched Domain Pre-Training Model and Adversarial Training

As important geological data,a geological report contains rich expert and geological knowledge,but the challenge facing current research into geological knowledge extraction and mining is how to render accurate understanding of geological reports guided by domain knowledge.While generic named entity recognition models/tools can be utilized for the processing of geoscience reports/documents,their effectiveness is hampered by a dearth of domain-specific knowledge,which in turn leads to a pronounced decline in recognition accuracy.This study summarizes six types of typical geological entities,with reference to the ontological system of geological domains and builds a high quality corpus for the task of geological named entity recognition(GNER).In addition,Geo Wo BERT-adv BGP(Geological Word-base BERTadversarial training Bi-directional Long Short-Term Memory Global Pointer)is proposed to address the issues of ambiguity,diversity and nested entities for the geological entities.The model first uses the fine-tuned word granularitybased pre-training model Geo Wo BERT(Geological Word-base BERT)and combines the text features that are extracted using the Bi LSTM(Bi-directional Long Short-Term Memory),followed by an adversarial training algorithm to improve the robustness of the model and enhance its resistance to interference,the decoding finally being performed using a global association pointer algorithm.The experimental results show that the proposed model for the constructed dataset achieves high performance and is capable of mining the rich geological information.

Classification of Conversational Sentences Using an Ensemble Pre-Trained Language Model with the Fine-Tuned Parameter

Sentence classification is the process of categorizing a sentence based on the context of the sentence.Sentence categorization requires more semantic highlights than other tasks,such as dependence parsing,which requires more syntactic elements.Most existing strategies focus on the general semantics of a conversation without involving the context of the sentence,recognizing the progress and comparing impacts.An ensemble pre-trained language model was taken up here to classify the conversation sentences from the conversation corpus.The conversational sentences are classified into four categories:information,question,directive,and commission.These classification label sequences are for analyzing the conversation progress and predicting the pecking order of the conversation.Ensemble of Bidirectional Encoder for Representation of Transformer(BERT),Robustly Optimized BERT pretraining Approach(RoBERTa),Generative Pre-Trained Transformer(GPT),DistilBERT and Generalized Autoregressive Pretraining for Language Understanding(XLNet)models are trained on conversation corpus with hyperparameters.Hyperparameter tuning approach is carried out for better performance on sentence classification.This Ensemble of Pre-trained Language Models with a Hyperparameter Tuning(EPLM-HT)system is trained on an annotated conversation dataset.The proposed approach outperformed compared to the base BERT,GPT,DistilBERT and XLNet transformer models.The proposed ensemble model with the fine-tuned parameters achieved an F1_score of 0.88.

Adapter Based on Pre-Trained Language Models for Classification of Medical Text

We present an approach to classify medical text at a sentence level automatically.Given the inherent complexity of medical text classification,we employ adapters based on pre-trained language models to extract information from medical text,facilitating more accurate classification while minimizing the number of trainable parameters.Extensive experiments conducted on various datasets demonstrate the effectiveness of our approach.

A Modified CycleGAN for Multi-Organ Ultrasound Image Enhancement via Unpaired Pre-Training

Handheld ultrasound devices are known for their portability and affordability,making them widely utilized in underdeveloped areas and community healthcare for rapid diagnosis and early screening.However,the image quality of handheld ultrasound devices is not always satisfactory due to the limited equipment size,which hinders accurate diagnoses by doctors.At the same time,paired ultrasound images are difficult to obtain from the clinic because imaging process is complicated.Therefore,we propose a modified cycle generative adversarial network(cycleGAN) for ultrasound image enhancement from multiple organs via unpaired pre-training.We introduce an ultrasound image pre-training method that does not require paired images,alleviating the requirement for large-scale paired datasets.We also propose an enhanced block with different structures in the pre-training and fine-tuning phases,which can help achieve the goals of different training phases.To improve the robustness of the model,we add Gaussian noise to the training images as data augmentation.Our approach is effective in obtaining the best quantitative evaluation results using a small number of parameters and less training costs to improve the quality of handheld ultrasound devices.

Knowledge Enhanced Pre-Training Model for Vision-Language-Navigation Task

Vision-Language-Navigation(VLN) task is a cross-modality task that combines natural language processing and computer vision. This task requires the agent to automatically move to the destination according to the natural language instruction and the observed surrounding visual information. To make the best decision, in every step during the navigation, the agent should pay more attention to understanding the objects, the object attributes, and the object relationships. But most current methods process all received textual and visual information equally. Therefore, this paper integrates more detailed semantic connections between visual and textual information through three pre-training tasks(object prediction, object attributes prediction, and object relationship prediction). The model will learn better fusion representation and alignment between these two types of information to improve the success rate(SR) and generalization. The experiments show that compared with the former baseline models, the SR on the unseen validation set(Val Unseen) increased by 7%, and the SR weighted by path length(SPL) increased by 7%;the SR on the test set(Test) increased 4%, SPL increased by 3%.

Enhancing Relational Triple Extraction in Specific Domains:Semantic Enhancement and Synergy of Large Language Models and Small Pre-Trained Language Models

In the process of constructing domain-specific knowledge graphs,the task of relational triple extraction plays a critical role in transforming unstructured text into structured information.Existing relational triple extraction models facemultiple challenges when processing domain-specific data,including insufficient utilization of semantic interaction information between entities and relations,difficulties in handling challenging samples,and the scarcity of domain-specific datasets.To address these issues,our study introduces three innovative components:Relation semantic enhancement,data augmentation,and a voting strategy,all designed to significantly improve the model’s performance in tackling domain-specific relational triple extraction tasks.We first propose an innovative attention interaction module.This method significantly enhances the semantic interaction capabilities between entities and relations by integrating semantic information fromrelation labels.Second,we propose a voting strategy that effectively combines the strengths of large languagemodels(LLMs)and fine-tuned small pre-trained language models(SLMs)to reevaluate challenging samples,thereby improving the model’s adaptability in specific domains.Additionally,we explore the use of LLMs for data augmentation,aiming to generate domain-specific datasets to alleviate the scarcity of domain data.Experiments conducted on three domain-specific datasets demonstrate that our model outperforms existing comparative models in several aspects,with F1 scores exceeding the State of the Art models by 2%,1.6%,and 0.6%,respectively,validating the effectiveness and generalizability of our approach.

Aquaporin-4-IgG-seropositive neuromyelitis optica spectrum disorders:progress of experimental models based on disease pathogenesis

Neuromyelitis optica spectrum disorders are neuroinflammatory demyelinating disorders that lead to permanent visual loss and motor dysfunction.To date,no effective treatment exists as the exact causative mechanism remains unknown.Therefore,experimental models of neuromyelitis optica spectrum disorders are essential for exploring its pathogenesis and in screening for therapeutic targets.Since most patients with neuromyelitis optica spectrum disorders are seropositive for IgG autoantibodies against aquaporin-4,which is highly expressed on the membrane of astrocyte endfeet,most current experimental models are based on aquaporin-4-IgG that initially targets astrocytes.These experimental models have successfully simulated many pathological features of neuromyelitis optica spectrum disorders,such as aquaporin-4 loss,astrocytopathy,granulocyte and macrophage infiltration,complement activation,demyelination,and neuronal loss;however,they do not fully capture the pathological process of human neuromyelitis optica spectrum disorders.In this review,we summarize the currently known pathogenic mechanisms and the development of associated experimental models in vitro,ex vivo,and in vivo for neuromyelitis optica spectrum disorders,suggest potential pathogenic mechanisms for further investigation,and provide guidance on experimental model choices.In addition,this review summarizes the latest information on pathologies and therapies for neuromyelitis optica spectrum disorders based on experimental models of aquaporin-4-IgG-seropositive neuromyelitis optica spectrum disorders,offering further therapeutic targets and a theoretical basis for clinical trials.

A Novel Optimization Scheme for Named Entity Recognition with Pre-trained Language Models

Named Entity Recognition(NER)is crucial for extracting structured information from text.While traditional methods rely on rules,Conditional Random Fields(CRFs),or deep learning,the advent of large-scale Pre-trained Language Models(PLMs)offers new possibilities.PLMs excel at contextual learning,potentially simplifying many natural language processing tasks.However,their application to NER remains underexplored.This paper investigates leveraging the GPT-3 PLM for NER without fine-tuning.We propose a novel scheme that utilizes carefully crafted templates and context examples selected based on semantic similarity.Our experimental results demonstrate the feasibility of this approach,suggesting a promising direction for harnessing PLMs in NER.

A Phonetic-Semantic Pre-Training Model for Robust Speech Recognition

Robustness is a long-standing challenge for automatic speech recognition(ASR)as the applied environment of any ASR system faces much noisier speech samples than clean training corpora.However,it is impractical to annotate every types of noisy environments.In this work,we propose a novel phonetic-semantic pre-training(PSP)framework that allows a model to effectively improve the performance of ASR against practical noisy environments via seamlessly integrating pre-training,self-supervised learning,and fine-tuning.In particular,there are three fundamental stages in PSP.First,pre-train the phone-to-word transducer(PWT)to map the generated phone sequence to the target text using only unpaired text data;second,continue training the PWT on more complex data generated from an empirical phone-perturbation heuristic,in additional to self-supervised signals by recovering the tainted phones;and third,fine-tune the resultant PWT with real world speech data.We perform experiments on two real-life datasets collected from industrial scenarios and synthetic noisy datasets,which show that the PSP effectively improves the traditional ASR pipeline with relative character error rate(CER)reductions of 28.63%and 26.38%,respectively,in two real-life datasets.It also demonstrates its robustness against synthetic highly noisy speech datasets.

Exploiting fly models to investigate rare human neurological disorders

Rare neurological diseases,while individually are rare,collectively impact millions globally,leading to diverse and often severe neurological symptoms.Often attributed to genetic mutations that disrupt protein function or structure,understanding their genetic basis is crucial for accurate diagnosis and targeted therapies.To investigate the underlying pathogenesis of these conditions,researchers often use non-mammalian model organisms,such as Drosophila(fruit flies),which is valued for their genetic manipulability,cost-efficiency,and preservation of genes and biological functions across evolutionary time.Genetic tools available in Drosophila,including CRISPR-Cas9,offer a means to manipulate gene expression,allowing for a deep exploration of the genetic underpinnings of rare neurological diseases.Drosophila boasts a versatile genetic toolkit,rapid generation turnover,and ease of large-scale experimentation,making it an invaluable resource for identifying potential drug candidates.Researchers can expose flies carrying disease-associated mutations to various compounds,rapidly pinpointing promising therapeutic agents for further investigation in mammalian models and,ultimately,clinical trials.In this comprehensive review,we explore rare neurological diseases where fly research has significantly contributed to our understanding of their genetic basis,pathophysiology,and potential therapeutic implications.We discuss rare diseases associated with both neuron-expressed and glial-expressed genes.Specific cases include mutations in CDK19 resulting in epilepsy and developmental delay,mutations in TIAM1 leading to a neurodevelopmental disorder with seizures and language delay,and mutations in IRF2BPL causing seizures,a neurodevelopmental disorder with regression,loss of speech,and abnormal movements.And we explore mutations in EMC1 related to cerebellar atrophy,visual impairment,psychomotor retardation,and gain-of-function mutations in ACOX1 causing Mitchell syndrome.Loss-of-function mutations in ACOX1 result in ACOX1 deficiency,characterized by very-long-chain fatty acid accumulation and glial degeneration.Notably,this review highlights how modeling these diseases in Drosophila has provided valuable insights into their pathophysiology,offering a platform for the rapid identification of potential therapeutic interventions.Rare neurological diseases involve a wide range of expression systems,and sometimes common phenotypes can be found among different genes that cause abnormalities in neurons or glia.Furthermore,mutations within the same gene may result in varying functional outcomes,such as complete loss of function,partial loss of function,or gain-of-function mutations.The phenotypes observed in patients can differ significantly,underscoring the complexity of these conditions.In conclusion,Drosophila represents an indispensable and cost-effective tool for investigating rare neurological diseases.By facilitating the modeling of these conditions,Drosophila contributes to a deeper understanding of their genetic basis,pathophysiology,and potential therapies.This approach accelerates the discovery of promising drug candidates,ultimately benefiting patients affected by these complex and understudied diseases.

A promising approach for quantifying focal stroke modeling and assessing stroke progression:optical resolution photoacoustic microscopy photothrombosis

To investigate the mechanisms underlying the onset and progression of ischemic stroke,some methods have been proposed that can simultaneously monitor and create embolisms in the animal cerebral cortex.However,these methods often require complex systems and the effect of age on cerebral embolism has not been adequately studied,although ischemic stroke is strongly age-related.In this study,we propose an optical-resolution photoacoustic microscopy-based visualized photothrombosis methodology to create and monitor ischemic stroke in mice simultaneously using a 532 nm pulsed laser.We observed the molding process in mice of different ages and presented age-dependent vascular embolism differentiation.Moreover,we integrated optical coherence tomography angiography to investigate age-associated trends in cerebrovascular variability following a stroke.Our imaging data and quantitative analyses underscore the differential cerebrovascular responses to stroke in mice of different ages,thereby highlighting the technique's potential for evaluating cerebrovascular health and unraveling age-related mechanisms involved in ischemic strokes.

Reduced mesencephalic astrocyte-derived neurotrophic factor expression by mutant androgen receptor contributes to neurodegeneration in a model of spinal and bulbar muscular atrophy pathology

Spinal and bulbar muscular atrophy is a neurodegenerative disease caused by extended CAG trinucleotide repeats in the androgen receptor gene,which encodes a ligand-dependent transcription facto r.The mutant androgen receptor protein,characterized by polyglutamine expansion,is prone to misfolding and forms aggregates in both the nucleus and cytoplasm in the brain in spinal and bulbar muscular atrophy patients.These aggregates alter protein-protein interactions and compromise transcriptional activity.In this study,we reported that in both cultured N2a cells and mouse brain,mutant androgen receptor with polyglutamine expansion causes reduced expression of mesencephalic astrocyte-de rived neurotrophic factor.Overexpressio n of mesencephalic astrocyte-derived neurotrophic factor amelio rated the neurotoxicity of mutant androgen receptor through the inhibition of mutant androgen receptor aggregation.Conversely.knocking down endogenous mesencephalic astrocyte-derived neurotrophic factor in the mouse brain exacerbated neuronal damage and mutant androgen receptor aggregation.Our findings suggest that inhibition of mesencephalic astrocyte-derived neurotrophic factor expression by mutant androgen receptor is a potential mechanism underlying neurodegeneration in spinal and bulbar muscular atrophy.

A Classification–Detection Approach of COVID-19 Based on Chest X-ray and CT by Using Keras Pre-Trained Deep Learning Models

The Coronavirus Disease 2019(COVID-19)is wreaking havoc around the world,bring out that the enormous pressure on national health and medical staff systems.One of the most effective and critical steps in the fight against COVID-19,is to examine the patient’s lungs based on the Chest X-ray and CT generated by radiation imaging.In this paper,five keras-related deep learning models:ResNet50,InceptionResNetV2,Xception,transfer learning and pre-trained VGGNet16 is applied to formulate an classification-detection approaches of COVID-19.Two benchmark methods SVM(Support Vector Machine),CNN(Conventional Neural Networks)are provided to compare with the classification-detection approaches based on the performance indicators,i.e.,precision,recall,F1 scores,confusion matrix,classification accuracy and three types of AUC(Area Under Curve).The highest classification accuracy derived by classification-detection based on 5857 Chest X-rays and 767 Chest CTs are respectively 84%and 75%,which shows that the keras-related deep learning approaches facilitate accurate and effective COVID-19-assisted detection.

采用STAMP-24Model的多组织事故分析

安全生产事故往往由多组织交互、多因素耦合造成,事故原因涉及多个组织。为预防和遏制多组织生产安全事故的发生,基于系统理论事故建模与过程模型(Systems-Theory Accident Modeling and Process,STAMP)、24Model,构建一种用于多组织事故分析的方法,并以青岛石油爆炸事故为例进行事故原因分析。结果显示:STAMP-24Model可以分组织,分层次且有效、全面、详细地分析涉及多个组织的事故原因,探究多组织之间的交互关系;对事故进行动态演化分析,可得到各组织不安全动作耦合关系与形成的事故失效链及管控失效路径,进而为预防多组织事故提供思路和参考。

基于改进24Model-ISM-SNA建筑工人不安全行为关联路径研究

建筑施工现场环境复杂,为有效控制不安全行为发生,基于行为安全“2-4”模型对360份具有代表性的建筑安全事故调查报告进行分析,提取出22个不安全行为的主要影响因素。利用灰色关联分析方法(GRA)改进的集成ISM-SNA模型,将不安全行为风险因素划分为表层、过渡层与深层,然后对风险因素进行可视化分析、中心度分析及凝聚子群分析,揭示了各致因因素间的关联关系和传导路径。结果表明,建筑工人不安全行为影响因素可划分成7级3阶的多级递阶结构,安全意识、现场监管、外部环境是建筑工人不安全行为的关键影响因素,同时现场监管和隐患排查到位能有效降低不安全行为的发生。

基于24Model的地铁内涝事故原因分析与评估

为降低地铁内涝事故灾害风险,基于事故致因“2-4”模型,分析了地铁内涝事故致灾因子,采用层次分析法构建了地铁内涝事故原因分析指标体系,确定了各风险因子的权重,并利用模糊综合评价法对地铁内涝事故进行了定量评估,识别出关键的影响因素。结果表明:地铁内涝事故一级指标中不安全动作与物态因素最重要,其中影响最大的指标包括擅自更改建筑设计、未按照要求检查水位情况、未及时排水、出入口不符合防汛标准等因素;习惯性不安全行为的权重位居第二,表明该指标因素较为重要,同时安全管理体系得分位居第二,表明该指标因素较易发生。对关键指标采取防范措施,可有效降低风险,从而减少地铁内涝事故的发生。

基于24Model-D-ISM的地铁站火灾疏散影响因素研究

为预防地铁站火灾事故,深入了解地铁站火灾人员疏散影响因素间的内在联系与层次结构,基于第6版“2-4”模型(24Model)分析63起地铁站火灾疏散事故,充分考虑各个因素之间的交互作用,提取19个影响地铁站人员疏散的关键因素,建立地铁站火灾人员疏散影响因素指标体系;采用算子客观赋权法(C-OWA)改进决策试验与评价实验法(DEMATEL),确定地铁站火灾人员疏散的重要影响因素;在此基础上,采用解释结构模型(ISM)分析各个因素间的层次结构及相互作用路径,构建地铁站火灾人员疏散影响因素的多级递阶结构模型。研究结果表明:疏散引导、恐慌从众行为、人员拥挤为地铁站火灾人员疏散的关键影响因素;地铁站火灾人员疏散受表层因素、中间层因素、深层因素共同作用的影响,其中,疏散教育与培训、设施维护与检查、疏散预案等因素是根源影响因素,重视根源影响因素的改善有利于从本质上预防和控制事故的发生。

Effective distributed convolutional neural network architecture for remote sensing images target classification with a pre-training approach

How to recognize targets with similar appearances from remote sensing images(RSIs) effectively and efficiently has become a big challenge. Recently, convolutional neural network(CNN) is preferred in the target classification due to the powerful feature representation ability and better performance. However,the training and testing of CNN mainly rely on single machine.Single machine has its natural limitation and bottleneck in processing RSIs due to limited hardware resources and huge time consuming. Besides, overfitting is a challenge for the CNN model due to the unbalance between RSIs data and the model structure.When a model is complex or the training data is relatively small,overfitting occurs and leads to a poor predictive performance. To address these problems, a distributed CNN architecture for RSIs target classification is proposed, which dramatically increases the training speed of CNN and system scalability. It improves the storage ability and processing efficiency of RSIs. Furthermore,Bayesian regularization approach is utilized in order to initialize the weights of the CNN extractor, which increases the robustness and flexibility of the CNN model. It helps prevent the overfitting and avoid the local optima caused by limited RSI training images or the inappropriate CNN structure. In addition, considering the efficiency of the Na¨?ve Bayes classifier, a distributed Na¨?ve Bayes classifier is designed to reduce the training cost. Compared with other algorithms, the proposed system and method perform the best and increase the recognition accuracy. The results show that the distributed system framework and the proposed algorithms are suitable for RSIs target classification tasks.

24Model与LCM原因因素定义对比研究

为探究损失致因模型(LCM)原因因素定义与事故致因“2-4”模型(24Model)存在的异同和优缺点,梳理2个模型各层面原因和结果的定义,对比定义内容及其对事故原因分析等安全实务的指导作用,并以一起瓦斯爆炸事故为例加以实证分析,获得二者分析结果之间的差异。研究结果表明:LCM是首个将管理因素纳入事故致因分析的一维事件序列模型,可明确各层面原因因素的定义和因素间的逻辑关系,但部分定义存在交叉重复的问题,并没有揭示安全工作指导思想等深层次事故致因因素;24Model作为系统性事故致因模型,对各类因素的定义均以组织为主体,描述事件、事故、安全的概念内涵,划分个体安全动作、安全能力和组织安全管理体系的类别并给出含义解析,探究组织安全文化层面的问题并以32个元素体现;2个模型的事故原因分析方法均建立在对各层级原因因素定义的基础上,并适用于模型理论体系本身。

Construction and application of knowledge graph for grid dispatch fault handling based on pre-trained model

With the construction of new power systems,the power grid has become extremely large,with an increasing proportion of new energy and AC/DC hybrid connections.The dynamic characteristics and fault patterns of the power grid are complex;additionally,power grid control is difficult,operation risks are high,and the task of fault handling is arduous.Traditional power-grid fault handling relies primarily on human experience.The difference in and lack of knowledge reserve of control personnel restrict the accuracy and timeliness of fault handling.Therefore,this mode of operation is no longer suitable for the requirements of new systems.Based on the multi-source heterogeneous data of power grid dispatch,this paper proposes a joint entity–relationship extraction method for power-grid dispatch fault processing based on a pre-trained model,constructs a knowledge graph of power-grid dispatch fault processing and designs,and develops a fault-processing auxiliary decision-making system based on the knowledge graph.It was applied to study a provincial dispatch control center,and it effectively improved the accident processing ability and intelligent level of accident management and control of the power grid.