Enhancing Relational Triple Extraction in Specific Domains:Semantic Enhancement and Synergy of Large Language Models and Small Pre-Trained Language Models

In the process of constructing domain-specific knowledge graphs,the task of relational triple extraction plays a critical role in transforming unstructured text into structured information.Existing relational triple extraction models facemultiple challenges when processing domain-specific data,including insufficient utilization of semantic interaction information between entities and relations,difficulties in handling challenging samples,and the scarcity of domain-specific datasets.To address these issues,our study introduces three innovative components:Relation semantic enhancement,data augmentation,and a voting strategy,all designed to significantly improve the model’s performance in tackling domain-specific relational triple extraction tasks.We first propose an innovative attention interaction module.This method significantly enhances the semantic interaction capabilities between entities and relations by integrating semantic information fromrelation labels.Second,we propose a voting strategy that effectively combines the strengths of large languagemodels(LLMs)and fine-tuned small pre-trained language models(SLMs)to reevaluate challenging samples,thereby improving the model’s adaptability in specific domains.Additionally,we explore the use of LLMs for data augmentation,aiming to generate domain-specific datasets to alleviate the scarcity of domain data.Experiments conducted on three domain-specific datasets demonstrate that our model outperforms existing comparative models in several aspects,with F1 scores exceeding the State of the Art models by 2%,1.6%,and 0.6%,respectively,validating the effectiveness and generalizability of our approach.

Large animal models for Huntington's disease research

Huntington'sdisease(HD)isahereditary neurodegenerative disorder for which there is currently no effectivetreatmentavailable.Consequently,the development of appropriate disease models is critical to thoroughly investigate disease progression.The genetic basis of HD involves the abnormal expansion of CAG repeats in the huntingtin(HTT)gene,leading to the expansion of a polyglutamine repeat in the HTT protein.Mutant HTT carrying the expanded polyglutamine repeat undergoes misfolding and forms aggregates in the brain,which precipitate selective neuronal loss in specific brain regions.Animal models play an important role in elucidating the pathogenesis of neurodegenerative disorders such as HD and in identifying potential therapeutic targets.Due to the marked species differences between rodents and larger animals,substantial efforts have been directed toward establishing large animal models for HD research.These models are pivotal for advancing the discovery of novel therapeutic targets,enhancing effective drug delivery methods,and improving treatment outcomes.We have explored the advantages of utilizing large animal models,particularly pigs,in previous reviews.Since then,however,significant progress has been made in developing more sophisticated animal models that faithfully replicate the typical pathology of HD.In the current review,we provide a comprehensive overview of large animal models of HD,incorporating recent findings regarding the establishment of HD knock-in(KI)pigs and their genetic therapy.We also explore the utilization of large animal models in HD research,with a focus on sheep,non-human primates(NHPs),and pigs.Our objective is to provide valuable insights into the application of these large animal models for the investigation and treatment of neurodegenerative disorders.

Research status and application of artificial intelligence large models in the oil and gas industry

This article elucidates the concept of large model technology,summarizes the research status of large model technology both domestically and internationally,provides an overview of the application status of large models in vertical industries,outlines the challenges and issues confronted in applying large models in the oil and gas sector,and offers prospects for the application of large models in the oil and gas industry.The existing large models can be briefly divided into three categories:large language models,visual large models,and multimodal large models.The application of large models in the oil and gas industry is still in its infancy.Based on open-source large language models,some oil and gas enterprises have released large language model products using methods like fine-tuning and retrieval augmented generation.Scholars have attempted to develop scenario-specific models for oil and gas operations by using visual/multimodal foundation models.A few researchers have constructed pre-trained foundation models for seismic data processing and interpretation,as well as core analysis.The application of large models in the oil and gas industry faces challenges such as current data quantity and quality being difficult to support the training of large models,high research and development costs,and poor algorithm autonomy and control.The application of large models should be guided by the needs of oil and gas business,taking the application of large models as an opportunity to improve data lifecycle management,enhance data governance capabilities,promote the construction of computing power,strengthen the construction of“artificial intelligence+energy”composite teams,and boost the autonomy and control of large model technology.

Evaluating the role of large language models in inflammatory bowel disease patient information

This letter evaluates the article by Gravina et al on ChatGPT’s potential in providing medical information for inflammatory bowel disease patients.While promising,it highlights the need for advanced techniques like reasoning+action and retrieval-augmented generation to improve accuracy and reliability.Emphasizing that simple question and answer testing is insufficient,it calls for more nuanced evaluation methods to truly gauge large language models’capabilities in clinical applications.

Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models

Since the 1950s,when the Turing Test was introduced,there has been notable progress in machine language intelligence.Language modeling,crucial for AI development,has evolved from statistical to neural models over the last two decades.Recently,transformer-based Pre-trained Language Models(PLM)have excelled in Natural Language Processing(NLP)tasks by leveraging large-scale training corpora.Increasing the scale of these models enhances performance significantly,introducing abilities like context learning that smaller models lack.The advancement in Large Language Models,exemplified by the development of ChatGPT,has made significant impacts both academically and industrially,capturing widespread societal interest.This survey provides an overview of the development and prospects from Large Language Models(LLM)to Large Multimodal Models(LMM).It first discusses the contributions and technological advancements of LLMs in the field of natural language processing,especially in text generation and language understanding.Then,it turns to the discussion of LMMs,which integrates various data modalities such as text,images,and sound,demonstrating advanced capabilities in understanding and generating cross-modal content,paving new pathways for the adaptability and flexibility of AI systems.Finally,the survey highlights the prospects of LMMs in terms of technological development and application potential,while also pointing out challenges in data integration,cross-modal understanding accuracy,providing a comprehensive perspective on the latest developments in this field.

A large language model-powered literature review for high-angle annular dark field imaging

High-angle annular dark field(HAADF)imaging in scanning transmission electron microscopy(STEM)has become an indispensable tool in materials science due to its ability to offer sub-°A resolution and provide chemical information through Z-contrast.This study leverages large language models(LLMs)to conduct a comprehensive bibliometric analysis of a large amount of HAADF-related literature(more than 41000 papers).By using LLMs,specifically ChatGPT,we were able to extract detailed information on applications,sample preparation methods,instruments used,and study conclusions.The findings highlight the capability of LLMs to provide a new perspective into HAADF imaging,underscoring its increasingly important role in materials science.Moreover,the rich information extracted from these publications can be harnessed to develop AI models that enhance the automation and intelligence of electron microscopes.

DeBERTa-GRU: Sentiment Analysis for Large Language Model

Modern technological advancements have made social media an essential component of daily life.Social media allow individuals to share thoughts,emotions,and ideas.Sentiment analysis plays the function of evaluating whether the sentiment of the text is positive,negative,neutral,or any other personal emotion to understand the sentiment context of the text.Sentiment analysis is essential in business and society because it impacts strategic decision-making.Sentiment analysis involves challenges due to lexical variation,an unlabeled dataset,and text distance correlations.The execution time increases due to the sequential processing of the sequence models.However,the calculation times for the Transformer models are reduced because of the parallel processing.This study uses a hybrid deep learning strategy to combine the strengths of the Transformer and Sequence models while ignoring their limitations.In particular,the proposed model integrates the Decoding-enhanced with Bidirectional Encoder Representations from Transformers(BERT)attention(DeBERTa)and the Gated Recurrent Unit(GRU)for sentiment analysis.Using the Decoding-enhanced BERT technique,the words are mapped into a compact,semantic word embedding space,and the Gated Recurrent Unit model can capture the distance contextual semantics correctly.The proposed hybrid model achieves F1-scores of 97%on the Twitter Large Language Model(LLM)dataset,which is much higher than the performance of new techniques.

LKPNR: Large Language Models and Knowledge Graph for Personalized News Recommendation Framework

Accurately recommending candidate news to users is a basic challenge of personalized news recommendation systems.Traditional methods are usually difficult to learn and acquire complex semantic information in news texts,resulting in unsatisfactory recommendation results.Besides,these traditional methods are more friendly to active users with rich historical behaviors.However,they can not effectively solve the long tail problem of inactive users.To address these issues,this research presents a novel general framework that combines Large Language Models(LLM)and Knowledge Graphs(KG)into traditional methods.To learn the contextual information of news text,we use LLMs’powerful text understanding ability to generate news representations with rich semantic information,and then,the generated news representations are used to enhance the news encoding in traditional methods.In addition,multi-hops relationship of news entities is mined and the structural information of news is encoded using KG,thus alleviating the challenge of long-tail distribution.Experimental results demonstrate that compared with various traditional models,on evaluation indicators such as AUC,MRR,nDCG@5 and nDCG@10,the framework significantly improves the recommendation performance.The successful integration of LLM and KG in our framework has established a feasible way for achieving more accurate personalized news recommendation.Our code is available at https://github.com/Xuan-ZW/LKPNR.

Potential use of large language models for mitigating students’problematic social media use:ChatGPT as an example

The problematic use of social media has numerous negative impacts on individuals'daily lives,interpersonal relationships,physical and mental health,and more.Currently,there are few methods and tools to alleviate problematic social media,and their potential is yet to be fully realized.Emerging large language models(LLMs)are becoming increasingly popular for providing information and assistance to people and are being applied in many aspects of life.In mitigating problematic social media use,LLMs such as ChatGPT can play a positive role by serving as conversational partners and outlets for users,providing personalized information and resources,monitoring and intervening in problematic social media use,and more.In this process,we should recognize both the enormous potential and endless possibilities of LLMs such as ChatGPT,leveraging their advantages to better address problematic social media use,while also acknowledging the limitations and potential pitfalls of ChatGPT technology,such as errors,limitations in issue resolution,privacy and security concerns,and potential overreliance.When we leverage the advantages of LLMs to address issues in social media usage,we must adopt a cautious and ethical approach,being vigilant of the potential adverse effects that LLMs may have in addressing problematic social media use to better harness technology to serve individuals and society.

Evaluating Privacy Leakage and Memorization Attacks on Large Language Models (LLMs) in Generative AI Applications

The recent interest in the deployment of Generative AI applications that use large language models (LLMs) has brought to the forefront significant privacy concerns, notably the leakage of Personally Identifiable Information (PII) and other confidential or protected information that may have been memorized during training, specifically during a fine-tuning or customization process. We describe different black-box attacks from potential adversaries and study their impact on the amount and type of information that may be recovered from commonly used and deployed LLMs. Our research investigates the relationship between PII leakage, memorization, and factors such as model size, architecture, and the nature of attacks employed. The study utilizes two broad categories of attacks: PII leakage-focused attacks (auto-completion and extraction attacks) and memorization-focused attacks (various membership inference attacks). The findings from these investigations are quantified using an array of evaluative metrics, providing a detailed understanding of LLM vulnerabilities and the effectiveness of different attacks.

Security Vulnerability Analyses of Large Language Models (LLMs) through Extension of the Common Vulnerability Scoring System (CVSS) Framework

Large Language Models (LLMs) have revolutionized Generative Artificial Intelligence (GenAI) tasks, becoming an integral part of various applications in society, including text generation, translation, summarization, and more. However, their widespread usage emphasizes the critical need to enhance their security posture to ensure the integrity and reliability of their outputs and minimize harmful effects. Prompt injections and training data poisoning attacks are two of the most prominent vulnerabilities in LLMs, which could potentially lead to unpredictable and undesirable behaviors, such as biased outputs, misinformation propagation, and even malicious content generation. The Common Vulnerability Scoring System (CVSS) framework provides a standardized approach to capturing the principal characteristics of vulnerabilities, facilitating a deeper understanding of their severity within the security and AI communities. By extending the current CVSS framework, we generate scores for these vulnerabilities such that organizations can prioritize mitigation efforts, allocate resources effectively, and implement targeted security measures to defend against potential risks.

Case study on the mechanics of NPR anchor cable compensation for large deformation tunnel in soft rock in the Transverse Mountain area,China

A study was conducted to analyze the deformation mechanism of strongly weathered quartz schist in the Daliangshan Tunnel,located in the western Transverse Mountain area.A large deformation problem was experienced during the tunnel construction.To mitigate this problem,a support system was designed incorporating negative Poisson ratio(NPR)anchor cables with negative Poisson ratio effect.Physical model experiments,field experiments,and numerical simulation experiments were conducted to investigate the compensation mechanical behavior of NPR anchor cables.The large deformations of soft rocks in the Daliangshan Tunnel are caused by a high ground stress,a high degree of joint fracture development,and a high degree of surrounding rock fragmentation.A compensation mechanics support system combining long and short NPR anchor cables was suggested to provide sufficient counter-support force(approximately 350 kN)for the surrounding rock inside the tunnel.Comparing the NPR anchor cable support system with the original support system used in the Daliangshan tunnel showed that an NPR anchor cable support system,combining cables of 6.3 m and 10.3 m in length,effectively prevented convergence of surrounding rock deformation,and the integrated settlement convergence value remained below 300 mm.This study provides an effective scientific basis for resolving large deformation problems in deeply buried soft rocks in western transverse mountain areas.

The influence of inter-band rock on rib spalling in longwall panel with large mining height

In order to improve rib stability,failure criteria and instability mode of a thick coal seam with inter-band rock layer are analysed in this study.A three-dimensional mechanical model is established for the rib by considering the rock layer.A safety factor is defined foy the rib,and it is observed that the safety factor exhibits a positive correlation with the thickness and strength of the inter-band rock.A calculation method for determining critical parameters of the rock layer is presented to ensure the rib stability.It is revealed that incomplete propagation of the fracture at the hard rock constitutes a fundamental prerequisite for ensuring the rib stability.The influence of the position of the inter-band rock in the coal seam on failure mechanism of the rib was thoroughly investigated by developing a series of physical models for the rib at the face area.The best position for the inter-band rock in the coal seam is at a height of 1.5 m away from the roof line,which tends to provide a good stability state for the rib.For different inter-band rock positions,two ways of controlling rib by increasing supports stiffness and flexible grouting reinforcement are proposed.

Large animal models of cardiac ischemia-reperfusion injury:Where are we now?

Large animal models of cardiac ischemia-reperfusion are critical for evaluation of the efficacy of cardioprotective interventions prior to clinical translation.Nonetheless,current cardioprotective strategies/interventions formulated in preclinical cardiovascular research are often limited to small animal models,which are not transferable or reproducible in large animal models due to different factors such as:(i)complex and varied features of human ischemic cardiac disease(ICD),which are challenging to mimic in animal models,(ii)significant differences in surgical techniques applied,and(iii)differences in cardiovascular anatomy and physiology between small versus large animals.This article highlights the advantages and disadvantages of different large animal models of preclinical cardiac ischemic reperfusion injury(IRI),as well as the different methods used to induce and assess IRI,and the obstacles faced in using large animals for translational research in the settings of cardiac IR.

Classification of Conversational Sentences Using an Ensemble Pre-Trained Language Model with the Fine-Tuned Parameter

Sentence classification is the process of categorizing a sentence based on the context of the sentence.Sentence categorization requires more semantic highlights than other tasks,such as dependence parsing,which requires more syntactic elements.Most existing strategies focus on the general semantics of a conversation without involving the context of the sentence,recognizing the progress and comparing impacts.An ensemble pre-trained language model was taken up here to classify the conversation sentences from the conversation corpus.The conversational sentences are classified into four categories:information,question,directive,and commission.These classification label sequences are for analyzing the conversation progress and predicting the pecking order of the conversation.Ensemble of Bidirectional Encoder for Representation of Transformer(BERT),Robustly Optimized BERT pretraining Approach(RoBERTa),Generative Pre-Trained Transformer(GPT),DistilBERT and Generalized Autoregressive Pretraining for Language Understanding(XLNet)models are trained on conversation corpus with hyperparameters.Hyperparameter tuning approach is carried out for better performance on sentence classification.This Ensemble of Pre-trained Language Models with a Hyperparameter Tuning(EPLM-HT)system is trained on an annotated conversation dataset.The proposed approach outperformed compared to the base BERT,GPT,DistilBERT and XLNet transformer models.The proposed ensemble model with the fine-tuned parameters achieved an F1_score of 0.88.

Modeling and simulation of solvent behavior and temperature distribution within long stick propellants with large web thickness undergoing drying

Drying is a complicated physical process which involves simultaneous heat and mass transfer in the removal of solvents inside propellants.Inappropriate drying techniques may result in the formation of a hard skin layer near the surface to block the free access of most solvent through for long stick propellants with large web thickness,which lead to lower drying efficiency and worse drying quality.This study aims to gain a comprehensive understanding of drying process and clarify the mechanism of the blocked layer near the propellant surface.A new three-dimensional coupled heat and mass transfer(3D-CHMT)model was successfully developed under transient conditions.The drying experiment results show that the 3DCHMT model could be applied to describe the drying process well since the relative error of the content of solvent between simulation and experiment values is only 5.5%.The solvent behavior simulation demonstrates that the mass transfer process can be divided into super-fast(SF)and subsequent minorfast(MF)stages,and the SF stage is vital to the prevention of the blocked layer against the free access for solvent molecules inside propellant grains.The effective solvent diffusion coefficient(Deff)of the propellant surface initially increases from 3.4×10^(-6)to 5.3×10^(-6)m^(2)/s as the temperature increases,and then decreases to 4.1×10^(-8)m^(2)/s at 60-100 min.The value of Deffof surface between 0-1.4 mm has a unique trend of change compared with other regions,and it is much lower than that of the internal at100 min under simulation conditions.Meanwhile,the temperature of the propellant surface increases rapidly at the SF stage(0-100 min)and then very slowly thereafter.Both the evolution of Deffand temperature distribution demonstrate that the blocked layer near the propellant surface has been formed in the time period of approximately 0-100 min and its thickness is about 1.4 mm.To mitigate the formation of blocked layer and improve its drying quality of finial propellant products effectively,it should be initially dried at lower drying temperature(30-40℃)in 0-100 min and then dried at higher drying temperature(50-60℃)to reduce drying time for later drying process in double base gun propellants.The present results can provide theoretical guidance for drying process and optimization of drying parameters for long stick propellants with large web thickness.

Adapter Based on Pre-Trained Language Models for Classification of Medical Text

We present an approach to classify medical text at a sentence level automatically.Given the inherent complexity of medical text classification,we employ adapters based on pre-trained language models to extract information from medical text,facilitating more accurate classification while minimizing the number of trainable parameters.Extensive experiments conducted on various datasets demonstrate the effectiveness of our approach.

A phenomenological model for plastic flow behavior of rotating band material with a large temperature range

The plastic flow behavior of the rotating band material is investigated in this paper. The rotating band material is processed from H96 brass alloy, which is hardened to a much higher yield strength compared to the annealed one. The dynamically uniaxial compression behavior of the material is tested using the split Hopkinson pressure bar(SHPB) with temperature and strain rate ranging from 297 to 1073 K and500 to 3000 s^(-1), respectively, and a phenomenological plastic flow stress model is developed to describe the mechanical behavior of the material. The material is found to present noticeable temperature sensitivity and weak strain-rate sensitivity. The construction of the plastic flow stress model has two steps. Firstly, three univariate stress functions, taking plastic strain, plastic strain rate and temperature as independent variable, respectively, are proposed by fixing the other two variables. Then, as the three univariate functions describe the special cases of flow stress behavior under various conditions, the principle of stress compatibility is adopted to obtain the complete flow stress function. The numerical results show that the proposed plastic flow stress model is more suitable for the rotating band material than the existing well-known models.

Optimization models of stand structure and selective cutting cycle for large diameter trees of broadleaved forest in Changbai Mountain

The optimum models of harvesting yield and net profits of large diameter trees for broadleaved forest were developed, of which include matrix growth sub-model, harvesting cost and wood price sub-models, based on the data from Hongshi Forestry Bureau, in Changbai Mountain region, Jilin Province, China. The data were measured in 232 permanent sample plots. With the data of permanent sample plots, the parameters of transition probability and ingrowth models were estimated, and some models were compared and partly modified. During the simulation of stand structure, four factors such as largest diameter residual tree （LDT）, the ratio of the number of trees in a given diameter class to those in the next larger diameter class （q）, residual basal area （RBA） and selective cutting cycle （C） were considered. The simulation results showed that the optimum stand structure parameters for large diameter trees are as follows： q is 1.2, LDT is 46cm, RBA is larger than 26 m^2 and selective cutting cycle time （C） is between 10 and 20 years.

Large animal ischemic stroke models: replicating human stroke pathophysiology

The high morbidity and mortality rate of ischemic stroke in humans has led to the development of numerous animal models that replicate human stroke to further understand the underlying pathophysiology and to explore potential therapeutic interventions.Although promising therapeutics have been identified using these animal models,with most undergoing significant testing in rodent models,the vast majority of these interventions have failed in human clinical trials.This failure of preclinical translation highlights the critical need for better therapeutic assessment in more clinically relevant ischemic stroke animal models.Large animal models such as non-human primates,sheep,pigs,and dogs are likely more predictive of human responses and outcomes due to brain anatomy and physiology that are more similar to humans-potentially making large animal testing a key step in the stroke therapy translational pipeline.The objective of this review is to highlight key characteristics that potentially make these gyrencephalic,large animal ischemic stroke models more predictive by comparing pathophysiological responses,tissue-level changes,and model limitations.