Evaluating the role of large language models in inflammatory bowel disease patient information

This letter evaluates the article by Gravina et al on ChatGPT’s potential in providing medical information for inflammatory bowel disease patients.While promising,it highlights the need for advanced techniques like reasoning+action and retrieval-augmented generation to improve accuracy and reliability.Emphasizing that simple question and answer testing is insufficient,it calls for more nuanced evaluation methods to truly gauge large language models’capabilities in clinical applications.

Security Vulnerability Analyses of Large Language Models (LLMs) through Extension of the Common Vulnerability Scoring System (CVSS) Framework

Large Language Models (LLMs) have revolutionized Generative Artificial Intelligence (GenAI) tasks, becoming an integral part of various applications in society, including text generation, translation, summarization, and more. However, their widespread usage emphasizes the critical need to enhance their security posture to ensure the integrity and reliability of their outputs and minimize harmful effects. Prompt injections and training data poisoning attacks are two of the most prominent vulnerabilities in LLMs, which could potentially lead to unpredictable and undesirable behaviors, such as biased outputs, misinformation propagation, and even malicious content generation. The Common Vulnerability Scoring System (CVSS) framework provides a standardized approach to capturing the principal characteristics of vulnerabilities, facilitating a deeper understanding of their severity within the security and AI communities. By extending the current CVSS framework, we generate scores for these vulnerabilities such that organizations can prioritize mitigation efforts, allocate resources effectively, and implement targeted security measures to defend against potential risks.

Research status and application of artificial intelligence large models in the oil and gas industry

This article elucidates the concept of large model technology,summarizes the research status of large model technology both domestically and internationally,provides an overview of the application status of large models in vertical industries,outlines the challenges and issues confronted in applying large models in the oil and gas sector,and offers prospects for the application of large models in the oil and gas industry.The existing large models can be briefly divided into three categories:large language models,visual large models,and multimodal large models.The application of large models in the oil and gas industry is still in its infancy.Based on open-source large language models,some oil and gas enterprises have released large language model products using methods like fine-tuning and retrieval augmented generation.Scholars have attempted to develop scenario-specific models for oil and gas operations by using visual/multimodal foundation models.A few researchers have constructed pre-trained foundation models for seismic data processing and interpretation,as well as core analysis.The application of large models in the oil and gas industry faces challenges such as current data quantity and quality being difficult to support the training of large models,high research and development costs,and poor algorithm autonomy and control.The application of large models should be guided by the needs of oil and gas business,taking the application of large models as an opportunity to improve data lifecycle management,enhance data governance capabilities,promote the construction of computing power,strengthen the construction of“artificial intelligence+energy”composite teams,and boost the autonomy and control of large model technology.

Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models

Since the 1950s,when the Turing Test was introduced,there has been notable progress in machine language intelligence.Language modeling,crucial for AI development,has evolved from statistical to neural models over the last two decades.Recently,transformer-based Pre-trained Language Models(PLM)have excelled in Natural Language Processing(NLP)tasks by leveraging large-scale training corpora.Increasing the scale of these models enhances performance significantly,introducing abilities like context learning that smaller models lack.The advancement in Large Language Models,exemplified by the development of ChatGPT,has made significant impacts both academically and industrially,capturing widespread societal interest.This survey provides an overview of the development and prospects from Large Language Models(LLM)to Large Multimodal Models(LMM).It first discusses the contributions and technological advancements of LLMs in the field of natural language processing,especially in text generation and language understanding.Then,it turns to the discussion of LMMs,which integrates various data modalities such as text,images,and sound,demonstrating advanced capabilities in understanding and generating cross-modal content,paving new pathways for the adaptability and flexibility of AI systems.Finally,the survey highlights the prospects of LMMs in terms of technological development and application potential,while also pointing out challenges in data integration,cross-modal understanding accuracy,providing a comprehensive perspective on the latest developments in this field.

Enhancing Relational Triple Extraction in Specific Domains:Semantic Enhancement and Synergy of Large Language Models and Small Pre-Trained Language Models

In the process of constructing domain-specific knowledge graphs,the task of relational triple extraction plays a critical role in transforming unstructured text into structured information.Existing relational triple extraction models facemultiple challenges when processing domain-specific data,including insufficient utilization of semantic interaction information between entities and relations,difficulties in handling challenging samples,and the scarcity of domain-specific datasets.To address these issues,our study introduces three innovative components:Relation semantic enhancement,data augmentation,and a voting strategy,all designed to significantly improve the model’s performance in tackling domain-specific relational triple extraction tasks.We first propose an innovative attention interaction module.This method significantly enhances the semantic interaction capabilities between entities and relations by integrating semantic information fromrelation labels.Second,we propose a voting strategy that effectively combines the strengths of large languagemodels(LLMs)and fine-tuned small pre-trained language models(SLMs)to reevaluate challenging samples,thereby improving the model’s adaptability in specific domains.Additionally,we explore the use of LLMs for data augmentation,aiming to generate domain-specific datasets to alleviate the scarcity of domain data.Experiments conducted on three domain-specific datasets demonstrate that our model outperforms existing comparative models in several aspects,with F1 scores exceeding the State of the Art models by 2%,1.6%,and 0.6%,respectively,validating the effectiveness and generalizability of our approach.

LKPNR: Large Language Models and Knowledge Graph for Personalized News Recommendation Framework

Accurately recommending candidate news to users is a basic challenge of personalized news recommendation systems.Traditional methods are usually difficult to learn and acquire complex semantic information in news texts,resulting in unsatisfactory recommendation results.Besides,these traditional methods are more friendly to active users with rich historical behaviors.However,they can not effectively solve the long tail problem of inactive users.To address these issues,this research presents a novel general framework that combines Large Language Models(LLM)and Knowledge Graphs(KG)into traditional methods.To learn the contextual information of news text,we use LLMs’powerful text understanding ability to generate news representations with rich semantic information,and then,the generated news representations are used to enhance the news encoding in traditional methods.In addition,multi-hops relationship of news entities is mined and the structural information of news is encoded using KG,thus alleviating the challenge of long-tail distribution.Experimental results demonstrate that compared with various traditional models,on evaluation indicators such as AUC,MRR,nDCG@5 and nDCG@10,the framework significantly improves the recommendation performance.The successful integration of LLM and KG in our framework has established a feasible way for achieving more accurate personalized news recommendation.Our code is available at https://github.com/Xuan-ZW/LKPNR.

Potential use of large language models for mitigating students’problematic social media use:ChatGPT as an example

The problematic use of social media has numerous negative impacts on individuals'daily lives,interpersonal relationships,physical and mental health,and more.Currently,there are few methods and tools to alleviate problematic social media,and their potential is yet to be fully realized.Emerging large language models(LLMs)are becoming increasingly popular for providing information and assistance to people and are being applied in many aspects of life.In mitigating problematic social media use,LLMs such as ChatGPT can play a positive role by serving as conversational partners and outlets for users,providing personalized information and resources,monitoring and intervening in problematic social media use,and more.In this process,we should recognize both the enormous potential and endless possibilities of LLMs such as ChatGPT,leveraging their advantages to better address problematic social media use,while also acknowledging the limitations and potential pitfalls of ChatGPT technology,such as errors,limitations in issue resolution,privacy and security concerns,and potential overreliance.When we leverage the advantages of LLMs to address issues in social media usage,we must adopt a cautious and ethical approach,being vigilant of the potential adverse effects that LLMs may have in addressing problematic social media use to better harness technology to serve individuals and society.

Evaluating Privacy Leakage and Memorization Attacks on Large Language Models (LLMs) in Generative AI Applications

The recent interest in the deployment of Generative AI applications that use large language models (LLMs) has brought to the forefront significant privacy concerns, notably the leakage of Personally Identifiable Information (PII) and other confidential or protected information that may have been memorized during training, specifically during a fine-tuning or customization process. We describe different black-box attacks from potential adversaries and study their impact on the amount and type of information that may be recovered from commonly used and deployed LLMs. Our research investigates the relationship between PII leakage, memorization, and factors such as model size, architecture, and the nature of attacks employed. The study utilizes two broad categories of attacks: PII leakage-focused attacks (auto-completion and extraction attacks) and memorization-focused attacks (various membership inference attacks). The findings from these investigations are quantified using an array of evaluative metrics, providing a detailed understanding of LLM vulnerabilities and the effectiveness of different attacks.

A large language model-powered literature review for high-angle annular dark field imaging

High-angle annular dark field(HAADF)imaging in scanning transmission electron microscopy(STEM)has become an indispensable tool in materials science due to its ability to offer sub-°A resolution and provide chemical information through Z-contrast.This study leverages large language models(LLMs)to conduct a comprehensive bibliometric analysis of a large amount of HAADF-related literature(more than 41000 papers).By using LLMs,specifically ChatGPT,we were able to extract detailed information on applications,sample preparation methods,instruments used,and study conclusions.The findings highlight the capability of LLMs to provide a new perspective into HAADF imaging,underscoring its increasingly important role in materials science.Moreover,the rich information extracted from these publications can be harnessed to develop AI models that enhance the automation and intelligence of electron microscopes.

DeBERTa-GRU: Sentiment Analysis for Large Language Model

Modern technological advancements have made social media an essential component of daily life.Social media allow individuals to share thoughts,emotions,and ideas.Sentiment analysis plays the function of evaluating whether the sentiment of the text is positive,negative,neutral,or any other personal emotion to understand the sentiment context of the text.Sentiment analysis is essential in business and society because it impacts strategic decision-making.Sentiment analysis involves challenges due to lexical variation,an unlabeled dataset,and text distance correlations.The execution time increases due to the sequential processing of the sequence models.However,the calculation times for the Transformer models are reduced because of the parallel processing.This study uses a hybrid deep learning strategy to combine the strengths of the Transformer and Sequence models while ignoring their limitations.In particular,the proposed model integrates the Decoding-enhanced with Bidirectional Encoder Representations from Transformers(BERT)attention(DeBERTa)and the Gated Recurrent Unit(GRU)for sentiment analysis.Using the Decoding-enhanced BERT technique,the words are mapped into a compact,semantic word embedding space,and the Gated Recurrent Unit model can capture the distance contextual semantics correctly.The proposed hybrid model achieves F1-scores of 97%on the Twitter Large Language Model(LLM)dataset,which is much higher than the performance of new techniques.

Smaller & Smarter: Score-Driven Network Chaining of Smaller Language Models

With the continuous evolution and expanding applications of Large Language Models (LLMs), there has been a noticeable surge in the size of the emerging models. It is not solely the growth in model size, primarily measured by the number of parameters, but also the subsequent escalation in computational demands, hardware and software prerequisites for training, all culminating in a substantial financial investment as well. In this paper, we present novel techniques like supervision, parallelization, and scoring functions to get better results out of chains of smaller language models, rather than relying solely on scaling up model size. Firstly, we propose an approach to quantify the performance of a Smaller Language Models (SLM) by introducing a corresponding supervisor model that incrementally corrects the encountered errors. Secondly, we propose an approach to utilize two smaller language models (in a network) performing the same task and retrieving the best relevant output from the two, ensuring peak performance for a specific task. Experimental evaluations establish the quantitative accuracy improvements on financial reasoning and arithmetic calculation tasks from utilizing techniques like supervisor models (in a network of model scenario), threshold scoring and parallel processing over a baseline study.

The impact of ChatGPT on foreign language teaching and learning: Opportunities in education and research

The revolutionary online application ChatGPT has brought immense concerns to the education field.Foreign language teachers being some of those most reliant on writing assessments were among the most anxious,exacerbated by the extensive media coverage about the much-fantasized functionality of the chatbot.Hence,the article starts by elucidating the mechanisms,functions and common misconceptions about ChatGPT.Issues and risks associated with its usage are discussed,followed by an in-depth discussion of how the chatbot can be harnessed by learners and teachers.It is argued that ChatGPT offers major opportunities for teachers and education institutes to improve second/foreign language teaching and assessments,which similarly provided researchers with an array of research opportunities,especially towards a more personalized learning experience.

Translating radiology reports into plain language using ChatGPT and GPT-4 with prompt learning:results,limitations,and potential

The large language model called ChatGPT has drawn extensively attention because of its human-like expression and reasoning abilities.In this study,we investigate the feasibility of using ChatGPT in experiments on translating radiology reports into plain language for patients and healthcare providers so that they are educated for improved healthcare.Radiology reports from 62 low-dose chest computed tomography lung cancer screening scans and 76 brain magnetic resonance imaging metastases screening scans were collected in the first half of February for this study.According to the evaluation by radiologists,ChatGPT can successfully translate radiology reports into plain language with an average score of 4.27 in the five-point system with 0.08 places of information missing and 0.07 places of misinformation.In terms of the suggestions provided by ChatGPT,they are generally relevant such as keeping following-up with doctors and closely monitoring any symptoms,and for about 37%of 138 cases in total ChatGPT offers specific suggestions based on findings in the report.ChatGPT also presents some randomness in its responses with occasionally over-simplified or neglected information,which can be mitigated using a more detailed prompt.Furthermore,ChatGPT results are compared with a newly released large model GPT-4,showing that GPT-4 can significantly improve the quality of translated reports.Our results show that it is feasible to utilize large language models in clinical education,and further efforts are needed to address limitations and maximize their potential.

Large Knowledge Model:Perspectives and Challenges

Humankind's understanding of the world is fundamentally linked to our perception and cognition,with human languages serving as one of the major carriers of world knowledge.In this vein,Large Language Models(LLMs)like ChatGPT epitomize the pre-training of extensive,sequence-based world knowledge into neural networks,facilitating the processing and manipulation of this knowledge in a parametric space.This article explores large models through the lens of"knowledge".We initially investigate the role of symbolic knowledge such as Knowledge Graphs(KGs)in enhancing LLMs,covering aspects like knowledge-augmented language model,structure-inducing pretraining,knowledgeable prompts,structured CoT,knowledge editing,semantic tools for LLM and knowledgeable Al agents.Subsequently,we examine how LLMs can boost traditional symbolic knowledge bases,encompassing aspects like using LLM as KG builder and controller,structured knowledge pretraining,and LLM-enhanced symbolic reasoning.Considering the intricate nature of human knowledge,we advocate for the creation of Large Knowledge Models(LKM),specifically engineered to manage diversified spectrum of knowledge structures.This promising undertaking would entail several key challenges,such as disentangling knowledge base from language models,cognitive alignment with human knowledge,integration of perception and cognition,and building large commonsense models for interacting with physical world,among others.We finally propose a five-"A"principle to distinguish the concept of LKM.

A Communication Theory Perspective on Prompting Engineering Methods for Large Language Models

The springing up of large language models(LLMs)has shifted the community from single-task-orientated natural language processing(NLP)research to a holistic end-to-end multi-task learning paradigm.Along this line of research endeavors in the area,LLM-based prompting methods have attracted much attention,partially due to the technological advantages brought by prompt engineering(PE)as well as the underlying NLP principles disclosed by various prompting methods.Traditional supervised learning usually requires training a model based on labeled data and then making predictions.In contrast,PE methods directly use the powerful capabilities of existing LLMs(e.g.,GPT-3 and GPT-4)via composing appropriate prompts,especially under few-shot or zero-shot scenarios.Facing the abundance of studies related to the prompting and the ever-evolving nature of this field,this article aims to 1)illustrate a novel perspective to review existing PE methods within the well-established communication theory framework,2)facilitate a better/deeper understanding of developing trends of existing PE methods used in three typical tasks,and 3)shed light on promising research directions for future PE methods.

Evaluating ChatGPT’s Consciousness and Its Capability to Pass the Turing Test: A Comprehensive Analysis

This study explores the capabilities of ChatGPT, specifically in relation to consciousness and its performance in the Turing Test. The article begins by examining the diverse perspectives among both the cognitive and AI researchers regarding ChatGPT’s ability to pass the Turing Test. It introduces a hierarchical categorization of the test versions, suggesting that ChatGPT approaches success in the test, albeit primarily with na?ve users. Expert users, conversely, can easily identify its limitations. The paper presents various theories of consciousness, with a particular focus on the Integrated Information Theory proposed by Tononi. This theory serves as the framework for assessing ChatGPT’s level of consciousness. Through an evaluation based on the five axioms and theorems of IIT, the study finds that ChatGPT surpasses previous AI systems in certain aspects;however, ChatGPT significantly falls short of achieving a level of consciousness, particularly when compared to biological sentient beings. The paper concludes by emphasizing the importance of recognizing ChatGPT and similar generative AI models as highly advanced and intelligent tools, yet distinctly lacking the consciousness attributes found in advanced living organisms.

Harnessing AI-human synergy for deep learning research analysis in ophthalmology with large language models assisting humans

Background:Research innovations inocular disease screening,diagnosis,and management have been boosted by deep learning(DL)in the last decade.To assess historical research trends and current advances,we conducted an artificial intelligence(AI)-human hybrid analysis of publications on DL in ophthalmology.Methods:All DL-related articles in ophthalmology,which were published between 2012 and 2022 from Web of Science,were included.500 high-impact articles annotated with key research information were used to fine-tune a large language models(LLM)for reviewing medical literature and extracting information.After verifying the LLM's accuracy in extracting diseases and imaging modalities,we analyzed trend of DL in ophthalmology with 2535 articles.Results:Researchers using LLM for literature analysis were 70%(P=0.0001)faster than those who did not,while achieving comparable accuracy(97%versus 98%,P=0.7681).The field of DL in ophthalmology has grown 116%annually,paralleling trends of the broader DL domain.The publications focused mainly on diabetic retinopathy(P=0.0003),glaucoma(P=0.0011),and age-related macular diseases(P=0.0001)using retinal fundus photographs(FP,P=0.0015)and optical coherence tomography(OCT,P=0.0001).DL studies utilizing multimodal images have been growing,with FP and OCT combined being the most frequent.Among the 500 high-impact articles,laboratory studies constituted the majority at 65.3%.Notably,a discernible decline in model accuracy was observed when categorizing by study design,notwithstanding its statistical insignificance.Furthermore,43 publicly available ocular image datasets were summarized.Conclusion:This study has characterized the landscape of publications on DL in ophthalmology,by identifying the trends and breakthroughs among research topics and the fast-growing areas.This study provides an efficient framework for combined AI-human analysis to comprehensively assess the current status and future trends in the field.

Evaluating the accuracy and reproducibility of ChatGPT-4 in answering patient questions related to small intestinal bacterial overgrowth

BACKGROUND Small intestinal bacterial overgrowth(SIBO)poses diagnostic and treatment challenges due to its complex management and evolving guidelines.Patients often seek online information related to their health,prompting interest in large language models,like GPT-4,as potential sources of patient education.AIM To investigate ChatGPT-4's accuracy and reproducibility in responding to patient questions related to SIBO.METHODS A total of 27 patient questions related to SIBO were curated from professional societies,Facebook groups,and Reddit threads.Each question was entered into GPT-4 twice on separate days to examine reproducibility of accuracy on separate occasions.GPT-4 generated responses were independently evaluated for accuracy and reproducibility by two motility fellowship-trained gastroenterologists.A third senior fellowship-trained gastroenterologist resolved disagreements.Accuracy of responses were graded using the scale:(1)Comprehensive;(2)Correct but inadequate;(3)Some correct and some incorrect;or(4)Completely incorrect.Two responses were generated for every question to evaluate reproducibility in accuracy.RESULTS In evaluating GPT-4's effectiveness at answering SIBO-related questions,it provided responses with correct information to 18/27(66.7%)of questions,with 16/27(59.3%)of responses graded as comprehensive and 2/27(7.4%)responses graded as correct but inadequate.The model provided responses with incorrect information to 9/27(33.3%)of questions,with 4/27(14.8%)of responses graded as completely incorrect and 5/27(18.5%)of responses graded as mixed correct and incorrect data.Accuracy varied by question category,with questions related to“basic knowledge”achieving the highest proportion of comprehensive responses(90%)and no incorrect responses.On the other hand,the“treatment”related questions yielded the lowest proportion of comprehensive responses(33.3%)and highest percent of completely incorrect responses(33.3%).A total of 77.8%of questions yielded reproducible responses.CONCLUSION Though GPT-4 shows promise as a supplementary tool for SIBO-related patient education,the model requires further refinement and validation in subsequent iterations prior to its integration into patient care.

WirelessLLM:Empowering Large Language Models Towards Wireless Intelligence

The rapid evolution of wireless technologies and the growing complexity of network infrastructures necessitate a paradigm shift in how communication networks are designed,configured,and managed. Recent advancements in large language models (LLMs) have sparked interest in their potential to revolutionize wireless communication systems. However, existing studies on LLMs for wireless systems are limited to a direct application for telecom language understanding. To empower LLMs with knowledge and expertise in the wireless domain, this paper proposes WirelessLLM, a comprehensive framework for adapting and enhancing LLMs to address the unique challenges and requirements of wireless communication networks. We first identify three foundational principles that underpin WirelessLLM:knowledge alignment, knowledge fusion, and knowledge evolution. Then,we investigate the enabling technologies to build WirelessLLM, including prompt engineering, retrieval augmented generation, tool usage, multi-modal pre-training, and domain-specific fine-tuning. Moreover, we present three case studies to demonstrate the practical applicability and benefits of WirelessLLM for solving typical problems in wireless networks. Finally, we conclude this paper by highlighting key challenges and outlining potential avenues for future research.

LLM4CP:Adapting Large Language Models for Channel Prediction

Channel prediction is an effective approach for reducing the feedback or estimation overhead in massive multi-input multi-output (m-MIMO) systems. However, existing channel prediction methods lack precision due to model mismatch errors or network generalization issues. Large language models (LLMs) have demonstrated powerful modeling and generalization abilities, and have been successfully applied to cross-modal tasks, including the time series analysis. Leveraging the expressive power of LLMs, we propose a pre-trained LLM-empowered channel prediction(LLM4CP)method to predict the future downlink channel state information (CSI) sequence based on the historical uplink CSI sequence. We fine-tune the network while freezing most of the parameters of the pre-trained LLM for better cross-modality knowledge transfer. To bridge the gap between the channel data and the feature space of the LLM,preprocessor, embedding, and output modules are specifically tailored by taking into account unique channel characteristics. Simulations validate that the proposed method achieves state-of-the-art (SOTA) prediction performance on full-sample, few-shot, and generalization tests with low training and inference costs.