GeoNER:Geological Named Entity Recognition with Enriched Domain Pre-Training Model and Adversarial Training

As important geological data,a geological report contains rich expert and geological knowledge,but the challenge facing current research into geological knowledge extraction and mining is how to render accurate understanding of geological reports guided by domain knowledge.While generic named entity recognition models/tools can be utilized for the processing of geoscience reports/documents,their effectiveness is hampered by a dearth of domain-specific knowledge,which in turn leads to a pronounced decline in recognition accuracy.This study summarizes six types of typical geological entities,with reference to the ontological system of geological domains and builds a high quality corpus for the task of geological named entity recognition(GNER).In addition,Geo Wo BERT-adv BGP(Geological Word-base BERTadversarial training Bi-directional Long Short-Term Memory Global Pointer)is proposed to address the issues of ambiguity,diversity and nested entities for the geological entities.The model first uses the fine-tuned word granularitybased pre-training model Geo Wo BERT(Geological Word-base BERT)and combines the text features that are extracted using the Bi LSTM(Bi-directional Long Short-Term Memory),followed by an adversarial training algorithm to improve the robustness of the model and enhance its resistance to interference,the decoding finally being performed using a global association pointer algorithm.The experimental results show that the proposed model for the constructed dataset achieves high performance and is capable of mining the rich geological information.

Classification of Conversational Sentences Using an Ensemble Pre-Trained Language Model with the Fine-Tuned Parameter

Sentence classification is the process of categorizing a sentence based on the context of the sentence.Sentence categorization requires more semantic highlights than other tasks,such as dependence parsing,which requires more syntactic elements.Most existing strategies focus on the general semantics of a conversation without involving the context of the sentence,recognizing the progress and comparing impacts.An ensemble pre-trained language model was taken up here to classify the conversation sentences from the conversation corpus.The conversational sentences are classified into four categories:information,question,directive,and commission.These classification label sequences are for analyzing the conversation progress and predicting the pecking order of the conversation.Ensemble of Bidirectional Encoder for Representation of Transformer(BERT),Robustly Optimized BERT pretraining Approach(RoBERTa),Generative Pre-Trained Transformer(GPT),DistilBERT and Generalized Autoregressive Pretraining for Language Understanding(XLNet)models are trained on conversation corpus with hyperparameters.Hyperparameter tuning approach is carried out for better performance on sentence classification.This Ensemble of Pre-trained Language Models with a Hyperparameter Tuning(EPLM-HT)system is trained on an annotated conversation dataset.The proposed approach outperformed compared to the base BERT,GPT,DistilBERT and XLNet transformer models.The proposed ensemble model with the fine-tuned parameters achieved an F1_score of 0.88.

Adapter Based on Pre-Trained Language Models for Classification of Medical Text

We present an approach to classify medical text at a sentence level automatically.Given the inherent complexity of medical text classification,we employ adapters based on pre-trained language models to extract information from medical text,facilitating more accurate classification while minimizing the number of trainable parameters.Extensive experiments conducted on various datasets demonstrate the effectiveness of our approach.

A Modified CycleGAN for Multi-Organ Ultrasound Image Enhancement via Unpaired Pre-Training

Handheld ultrasound devices are known for their portability and affordability,making them widely utilized in underdeveloped areas and community healthcare for rapid diagnosis and early screening.However,the image quality of handheld ultrasound devices is not always satisfactory due to the limited equipment size,which hinders accurate diagnoses by doctors.At the same time,paired ultrasound images are difficult to obtain from the clinic because imaging process is complicated.Therefore,we propose a modified cycle generative adversarial network(cycleGAN) for ultrasound image enhancement from multiple organs via unpaired pre-training.We introduce an ultrasound image pre-training method that does not require paired images,alleviating the requirement for large-scale paired datasets.We also propose an enhanced block with different structures in the pre-training and fine-tuning phases,which can help achieve the goals of different training phases.To improve the robustness of the model,we add Gaussian noise to the training images as data augmentation.Our approach is effective in obtaining the best quantitative evaluation results using a small number of parameters and less training costs to improve the quality of handheld ultrasound devices.

Knowledge Enhanced Pre-Training Model for Vision-Language-Navigation Task

Vision-Language-Navigation(VLN) task is a cross-modality task that combines natural language processing and computer vision. This task requires the agent to automatically move to the destination according to the natural language instruction and the observed surrounding visual information. To make the best decision, in every step during the navigation, the agent should pay more attention to understanding the objects, the object attributes, and the object relationships. But most current methods process all received textual and visual information equally. Therefore, this paper integrates more detailed semantic connections between visual and textual information through three pre-training tasks(object prediction, object attributes prediction, and object relationship prediction). The model will learn better fusion representation and alignment between these two types of information to improve the success rate(SR) and generalization. The experiments show that compared with the former baseline models, the SR on the unseen validation set(Val Unseen) increased by 7%, and the SR weighted by path length(SPL) increased by 7%;the SR on the test set(Test) increased 4%, SPL increased by 3%.

Enhancing Relational Triple Extraction in Specific Domains:Semantic Enhancement and Synergy of Large Language Models and Small Pre-Trained Language Models

In the process of constructing domain-specific knowledge graphs,the task of relational triple extraction plays a critical role in transforming unstructured text into structured information.Existing relational triple extraction models facemultiple challenges when processing domain-specific data,including insufficient utilization of semantic interaction information between entities and relations,difficulties in handling challenging samples,and the scarcity of domain-specific datasets.To address these issues,our study introduces three innovative components:Relation semantic enhancement,data augmentation,and a voting strategy,all designed to significantly improve the model’s performance in tackling domain-specific relational triple extraction tasks.We first propose an innovative attention interaction module.This method significantly enhances the semantic interaction capabilities between entities and relations by integrating semantic information fromrelation labels.Second,we propose a voting strategy that effectively combines the strengths of large languagemodels(LLMs)and fine-tuned small pre-trained language models(SLMs)to reevaluate challenging samples,thereby improving the model’s adaptability in specific domains.Additionally,we explore the use of LLMs for data augmentation,aiming to generate domain-specific datasets to alleviate the scarcity of domain data.Experiments conducted on three domain-specific datasets demonstrate that our model outperforms existing comparative models in several aspects,with F1 scores exceeding the State of the Art models by 2%,1.6%,and 0.6%,respectively,validating the effectiveness and generalizability of our approach.

Comparative analysis of empirical and deep learning models for ionospheric sporadic E layer prediction

Sporadic E(Es)layers in the ionosphere are characterized by intense plasma irregularities in the E region at altitudes of 90-130 km.Because they can significantly influence radio communications and navigation systems,accurate forecasting of Es layers is crucial for ensuring the precision and dependability of navigation satellite systems.In this study,we present Es predictions made by an empirical model and by a deep learning model,and analyze their differences comprehensively by comparing the model predictions to satellite RO measurements and ground-based ionosonde observations.The deep learning model exhibited significantly better performance,as indicated by its high coefficient of correlation(r=0.87)with RO observations and predictions,than did the empirical model(r=0.53).This study highlights the importance of integrating artificial intelligence technology into ionosphere modelling generally,and into predicting Es layer occurrences and characteristics,in particular.

Dynamics of a Stochastic Epidemic Model with Age-group

A stochastic epidemic model with two age groups is established in this study,in which the susceptible(S),the exposed(E),the infected(I),the hospitalized(H)and the recovered(R)are involved within the total population,the aging rates between two age groups are set to be constant.The existence-and-uniqueness of global positive solution is firstly showed.Then,by constructing several appropriate Lyapunov functions and using the high-dimensional Itô’s formula,the sufficient conditions for the stochastic extinction and stochastic persistence of the exposed individuals and the infected individuals are obtained.The stochastic extinction indicator and the stochastic persistence indicator are less-valued expressions compared with the basic reproduction number.Meanwhile,the main results of this study are modified into multi-age groups.Furthermore,by using the surveillance data for Fujian Provincial Center for Disease Control and Prevention,Fuzhou COVID-19 epidemic is chosen to carry out the numerical simulations,which show that the age group of the population plays the vital role when studying infectious diseases.

Aquaporin-4-IgG-seropositive neuromyelitis optica spectrum disorders:progress of experimental models based on disease pathogenesis

Neuromyelitis optica spectrum disorders are neuroinflammatory demyelinating disorders that lead to permanent visual loss and motor dysfunction.To date,no effective treatment exists as the exact causative mechanism remains unknown.Therefore,experimental models of neuromyelitis optica spectrum disorders are essential for exploring its pathogenesis and in screening for therapeutic targets.Since most patients with neuromyelitis optica spectrum disorders are seropositive for IgG autoantibodies against aquaporin-4,which is highly expressed on the membrane of astrocyte endfeet,most current experimental models are based on aquaporin-4-IgG that initially targets astrocytes.These experimental models have successfully simulated many pathological features of neuromyelitis optica spectrum disorders,such as aquaporin-4 loss,astrocytopathy,granulocyte and macrophage infiltration,complement activation,demyelination,and neuronal loss;however,they do not fully capture the pathological process of human neuromyelitis optica spectrum disorders.In this review,we summarize the currently known pathogenic mechanisms and the development of associated experimental models in vitro,ex vivo,and in vivo for neuromyelitis optica spectrum disorders,suggest potential pathogenic mechanisms for further investigation,and provide guidance on experimental model choices.In addition,this review summarizes the latest information on pathologies and therapies for neuromyelitis optica spectrum disorders based on experimental models of aquaporin-4-IgG-seropositive neuromyelitis optica spectrum disorders,offering further therapeutic targets and a theoretical basis for clinical trials.

Solar flare forecasting based on a Fusion Model

Solar flare prediction is an important subject in the field of space weather.Deep learning technology has greatly promoted the development of this subject.In this study,we propose a novel solar flare forecasting model integrating Deep Residual Network(ResNet)and Support Vector Machine(SVM)for both≥C-class(C,M,and X classes)and≥M-class(M and X classes)flares.We collected samples of magnetograms from May 1,2010 to September 13,2018 from Space-weather Helioseismic and Magnetic Imager(HMI)Active Region Patches and then used a cross-validation method to obtain seven independent data sets.We then utilized five metrics to evaluate our fusion model,based on intermediate-output extracted by ResNet and SVM using the Gaussian kernel function.Our results show that the primary metric true skill statistics(TSS)achieves a value of 0.708±0.027 for≥C-class prediction,and of 0.758±0.042 for≥M-class prediction;these values indicate that our approach performs significantly better than those of previous studies.The metrics of our fusion model’s performance on the seven datasets indicate that the model is quite stable and robust,suggesting that fusion models that integrate an excellent baseline network with SVM can achieve improved performance in solar flare prediction.Besides,we also discuss the performance impact of architectural innovation in our fusion model.

A Novel Optimization Scheme for Named Entity Recognition with Pre-trained Language Models

Named Entity Recognition(NER)is crucial for extracting structured information from text.While traditional methods rely on rules,Conditional Random Fields(CRFs),or deep learning,the advent of large-scale Pre-trained Language Models(PLMs)offers new possibilities.PLMs excel at contextual learning,potentially simplifying many natural language processing tasks.However,their application to NER remains underexplored.This paper investigates leveraging the GPT-3 PLM for NER without fine-tuning.We propose a novel scheme that utilizes carefully crafted templates and context examples selected based on semantic similarity.Our experimental results demonstrate the feasibility of this approach,suggesting a promising direction for harnessing PLMs in NER.

A Phonetic-Semantic Pre-Training Model for Robust Speech Recognition

Robustness is a long-standing challenge for automatic speech recognition(ASR)as the applied environment of any ASR system faces much noisier speech samples than clean training corpora.However,it is impractical to annotate every types of noisy environments.In this work,we propose a novel phonetic-semantic pre-training(PSP)framework that allows a model to effectively improve the performance of ASR against practical noisy environments via seamlessly integrating pre-training,self-supervised learning,and fine-tuning.In particular,there are three fundamental stages in PSP.First,pre-train the phone-to-word transducer(PWT)to map the generated phone sequence to the target text using only unpaired text data;second,continue training the PWT on more complex data generated from an empirical phone-perturbation heuristic,in additional to self-supervised signals by recovering the tainted phones;and third,fine-tune the resultant PWT with real world speech data.We perform experiments on two real-life datasets collected from industrial scenarios and synthetic noisy datasets,which show that the PSP effectively improves the traditional ASR pipeline with relative character error rate(CER)reductions of 28.63%and 26.38%,respectively,in two real-life datasets.It also demonstrates its robustness against synthetic highly noisy speech datasets.

Landslide Susceptibility Mapping Using RBFN-Based Ensemble Machine Learning Models

This study was aimed to prepare landslide susceptibility maps for the Pithoragarh district in Uttarakhand,India,using advanced ensemble models that combined Radial Basis Function Networks(RBFN)with three ensemble learning techniques:DAGGING(DG),MULTIBOOST(MB),and ADABOOST(AB).This combination resulted in three distinct ensemble models:DG-RBFN,MB-RBFN,and AB-RBFN.Additionally,a traditional weighted method,Information Value(IV),and a benchmark machine learning(ML)model,Multilayer Perceptron Neural Network(MLP),were employed for comparison and validation.The models were developed using ten landslide conditioning factors,which included slope,aspect,elevation,curvature,land cover,geomorphology,overburden depth,lithology,distance to rivers and distance to roads.These factors were instrumental in predicting the output variable,which was the probability of landslide occurrence.Statistical analysis of the models’performance indicated that the DG-RBFN model,with an Area Under ROC Curve(AUC)of 0.931,outperformed the other models.The AB-RBFN model achieved an AUC of 0.929,the MB-RBFN model had an AUC of 0.913,and the MLP model recorded an AUC of 0.926.These results suggest that the advanced ensemble ML model DG-RBFN was more accurate than traditional statistical model,single MLP model,and other ensemble models in preparing trustworthy landslide susceptibility maps,thereby enhancing land use planning and decision-making.

Global Piecewise Analysis of HIV Model with Bi-Infectious Categories under Ordinary Derivative and Non-Singular Operator with Neural Network Approach

This study directs the discussion of HIV disease with a novel kind of complex dynamical generalized and piecewise operator in the sense of classical and Atangana Baleanu(AB)derivatives having arbitrary order.The HIV infection model has a susceptible class,a recovered class,along with a case of infection divided into three sub-different levels or categories and the recovered class.The total time interval is converted into two,which are further investigated for ordinary and fractional order operators of the AB derivative,respectively.The proposed model is tested separately for unique solutions and existence on bi intervals.The numerical solution of the proposed model is treated by the piece-wise numerical iterative scheme of Newtons Polynomial.The proposed method is established for piece-wise derivatives under natural order and non-singular Mittag-Leffler Law.The cross-over or bending characteristics in the dynamical system of HIV are easily examined by the aspect of this research having a memory effect for controlling the said disease.This study uses the neural network(NN)technique to obtain a better set of weights with low residual errors,and the epochs number is considered 1000.The obtained figures represent the approximate solution and absolute error which are tested with NN to train the data accurately.

Prognostic model for esophagogastric variceal rebleeding after endoscopic treatment in liver cirrhosis: A Chinese multicenter study

BACKGROUND Rebleeding after recovery from esophagogastric variceal bleeding(EGVB)is a severe complication that is associated with high rates of both incidence and mortality.Despite its clinical importance,recognized prognostic models that can effectively predict esophagogastric variceal rebleeding in patients with liver cirrhosis are lacking.AIM To construct and externally validate a reliable prognostic model for predicting the occurrence of esophagogastric variceal rebleeding.METHODS This study included 477 EGVB patients across 2 cohorts:The derivation cohort(n=322)and the validation cohort(n=155).The primary outcome was rebleeding events within 1 year.The least absolute shrinkage and selection operator was applied for predictor selection,and multivariate Cox regression analysis was used to construct the prognostic model.Internal validation was performed with bootstrap resampling.We assessed the discrimination,calibration and accuracy of the model,and performed patient risk stratification.RESULTS Six predictors,including albumin and aspartate aminotransferase concentrations,white blood cell count,and the presence of ascites,portal vein thrombosis,and bleeding signs,were selected for the rebleeding event prediction following endoscopic treatment(REPET)model.In predicting rebleeding within 1 year,the REPET model ex-hibited a concordance index of 0.775 and a Brier score of 0.143 in the derivation cohort,alongside 0.862 and 0.127 in the validation cohort.Furthermore,the REPET model revealed a significant difference in rebleeding rates(P<0.01)between low-risk patients and intermediate-to high-risk patients in both cohorts.CONCLUSION We constructed and validated a new prognostic model for variceal rebleeding with excellent predictive per-formance,which will improve the clinical management of rebleeding in EGVB patients.

High-throughput screening of CO_(2) cycloaddition MOF catalyst with an explainable machine learning model

The high porosity and tunable chemical functionality of metal-organic frameworks(MOFs)make it a promising catalyst design platform.High-throughput screening of catalytic performance is feasible since the large MOF structure database is available.In this study,we report a machine learning model for high-throughput screening of MOF catalysts for the CO_(2) cycloaddition reaction.The descriptors for model training were judiciously chosen according to the reaction mechanism,which leads to high accuracy up to 97%for the 75%quantile of the training set as the classification criterion.The feature contribution was further evaluated with SHAP and PDP analysis to provide a certain physical understanding.12,415 hypothetical MOF structures and 100 reported MOFs were evaluated under 100℃ and 1 bar within one day using the model,and 239 potentially efficient catalysts were discovered.Among them,MOF-76(Y)achieved the top performance experimentally among reported MOFs,in good agreement with the prediction.

Exploiting fly models to investigate rare human neurological disorders

Rare neurological diseases,while individually are rare,collectively impact millions globally,leading to diverse and often severe neurological symptoms.Often attributed to genetic mutations that disrupt protein function or structure,understanding their genetic basis is crucial for accurate diagnosis and targeted therapies.To investigate the underlying pathogenesis of these conditions,researchers often use non-mammalian model organisms,such as Drosophila(fruit flies),which is valued for their genetic manipulability,cost-efficiency,and preservation of genes and biological functions across evolutionary time.Genetic tools available in Drosophila,including CRISPR-Cas9,offer a means to manipulate gene expression,allowing for a deep exploration of the genetic underpinnings of rare neurological diseases.Drosophila boasts a versatile genetic toolkit,rapid generation turnover,and ease of large-scale experimentation,making it an invaluable resource for identifying potential drug candidates.Researchers can expose flies carrying disease-associated mutations to various compounds,rapidly pinpointing promising therapeutic agents for further investigation in mammalian models and,ultimately,clinical trials.In this comprehensive review,we explore rare neurological diseases where fly research has significantly contributed to our understanding of their genetic basis,pathophysiology,and potential therapeutic implications.We discuss rare diseases associated with both neuron-expressed and glial-expressed genes.Specific cases include mutations in CDK19 resulting in epilepsy and developmental delay,mutations in TIAM1 leading to a neurodevelopmental disorder with seizures and language delay,and mutations in IRF2BPL causing seizures,a neurodevelopmental disorder with regression,loss of speech,and abnormal movements.And we explore mutations in EMC1 related to cerebellar atrophy,visual impairment,psychomotor retardation,and gain-of-function mutations in ACOX1 causing Mitchell syndrome.Loss-of-function mutations in ACOX1 result in ACOX1 deficiency,characterized by very-long-chain fatty acid accumulation and glial degeneration.Notably,this review highlights how modeling these diseases in Drosophila has provided valuable insights into their pathophysiology,offering a platform for the rapid identification of potential therapeutic interventions.Rare neurological diseases involve a wide range of expression systems,and sometimes common phenotypes can be found among different genes that cause abnormalities in neurons or glia.Furthermore,mutations within the same gene may result in varying functional outcomes,such as complete loss of function,partial loss of function,or gain-of-function mutations.The phenotypes observed in patients can differ significantly,underscoring the complexity of these conditions.In conclusion,Drosophila represents an indispensable and cost-effective tool for investigating rare neurological diseases.By facilitating the modeling of these conditions,Drosophila contributes to a deeper understanding of their genetic basis,pathophysiology,and potential therapies.This approach accelerates the discovery of promising drug candidates,ultimately benefiting patients affected by these complex and understudied diseases.

A promising approach for quantifying focal stroke modeling and assessing stroke progression:optical resolution photoacoustic microscopy photothrombosis

To investigate the mechanisms underlying the onset and progression of ischemic stroke,some methods have been proposed that can simultaneously monitor and create embolisms in the animal cerebral cortex.However,these methods often require complex systems and the effect of age on cerebral embolism has not been adequately studied,although ischemic stroke is strongly age-related.In this study,we propose an optical-resolution photoacoustic microscopy-based visualized photothrombosis methodology to create and monitor ischemic stroke in mice simultaneously using a 532 nm pulsed laser.We observed the molding process in mice of different ages and presented age-dependent vascular embolism differentiation.Moreover,we integrated optical coherence tomography angiography to investigate age-associated trends in cerebrovascular variability following a stroke.Our imaging data and quantitative analyses underscore the differential cerebrovascular responses to stroke in mice of different ages,thereby highlighting the technique's potential for evaluating cerebrovascular health and unraveling age-related mechanisms involved in ischemic strokes.

Vertical gradients of neutral winds observed by ICON and estimated by the Horizontal Wind Model during the geomagnetic storm on August 26−28,2021

The Michelson Interferometer for Global High-resolution Thermospheric Imaging(MIGHTI)onboard the Ionospheric Connection Explorer(ICON)satellite offers the opportunity to investigate the altitude profile of thermospheric winds.In this study,we used the red-line measurements of MIGHTI to compare with the results estimated by Horizontal Wind Model 14(HWM14).The data selected included both the geomagnetic quiet period(December 2019 to August 2022)and the geomagnetic storm on August 26-28,2021.During the geomagnetic quiet period,the estimations of neutral winds from HWM14 showed relatively good agreement with the observations from ICON.According to the ICON observations,near the equator,zonal winds reverse from westward to eastward at around 06:00 local time(LT)at higher altitudes,and the stronger westward winds appear at later LTs at lower altitudes.At around 16:00 LT,eastward winds at 300 km reverse to westward,and vertical gradients of zonal winds similar to those at sunrise hours can be observed.In the middle latitudes,zonal winds reverse about 2-4 h earlier.Meridional winds vary more significantly than zonal winds with seasonal and latitudinal variations.According to the ICON observations,in the northern low latitudes,vertical reversals of meridional winds are found at 08:00-13:00 LT from 300 to 160 km and at around 18:00 LT from 300 to 200 km during the June solstice.Similar reversals of meridional winds are found at 04:00-07:00 LT from 300 to 160 km and at 22:00-02:00 LT from 270 to 200 km during the December solstice.In the southern low latitudes,meridional wind reversals occur at 08:00-11:00 LT from 200 to 160 km and at 21:00-02:00 LT from 300 to 200 km during the June solstice.During the December solstice,reversals of the meridional wind appear at 20:00-01:00 LT below 200 km and at 06:00-11:00 LT from 300 to 160 km.In the northern middle latitudes,the northward winds are dominant at 08:00-14:00 LT at 230 km during the June solstice.Northward winds persist until 16:00 LT at 160 and 300 km.During the December solstice,the northward winds are dominant from 06:00 to 21:00 LT.The vertical variations in neutral winds during the geomagnetic storm on August 26-28 were analyzed in detail.Both meridional and zonal winds during the active geomagnetic period observed by ICON show distinguishable vertical shear structures at different stages of the storm.On the dayside,during the main phase,the peak velocities of westward winds extend from a higher altitude to a lower altitude,whereas during the recovery phase,the peak velocities of the westward winds extend from lower altitudes to higher altitudes.The velocities of the southward winds are stronger at lower altitudes during the storm.These vertical structures of horizontal winds during the storm could not be reproduced by the HWM14 wind estimations,and the overall response to the storm of the horizontal winds in the low and middle latitudes is underestimated by HWM14.The ICON observations provide a good dataset for improving the HWM wind estimations in the middle and upper atmosphere,especially the vertical variations.

Rat models of frozen shoulder:Classification and evaluation

Frozen shoulder(FS),also known as adhesive capsulitis,is a condition that causes contraction and stiffness of the shoulder joint capsule.The main symptoms are per-sistent shoulder pain and a limited range of motion in all directions.These symp-toms and poor prognosis affect people's physical health and quality of life.Currently,the specific mechanisms of FS remain unclear,and there is variability in treatment methods and their efficacy.Additionally,the early symptoms of FS are difficult to distinguish from those of other shoulder diseases,complicating early diagnosis and treatment.Therefore,it is necessary to develop and utilize animal models to under-stand the pathogenesis of FS and to explore treatment strategies,providing insights into the prevention and treatment of human FS.This paper reviews the rat models available for FS research,including external immobilization models,surgical internal immobilization models,injection modeling models,and endocrine modeling models.It introduces the basic procedures for these models and compares and analyzes the advantages,disadvantages,and applicability of each modeling method.Finally,our paper summarizes the common methods for evaluating FS rat models.