Evaluating Privacy Leakage and Memorization Attacks on Large Language Models (LLMs) in Generative AI Applications

The recent interest in the deployment of Generative AI applications that use large language models (LLMs) has brought to the forefront significant privacy concerns, notably the leakage of Personally Identifiable Information (PII) and other confidential or protected information that may have been memorized during training, specifically during a fine-tuning or customization process. We describe different black-box attacks from potential adversaries and study their impact on the amount and type of information that may be recovered from commonly used and deployed LLMs. Our research investigates the relationship between PII leakage, memorization, and factors such as model size, architecture, and the nature of attacks employed. The study utilizes two broad categories of attacks: PII leakage-focused attacks (auto-completion and extraction attacks) and memorization-focused attacks (various membership inference attacks). The findings from these investigations are quantified using an array of evaluative metrics, providing a detailed understanding of LLM vulnerabilities and the effectiveness of different attacks.

Security Vulnerability Analyses of Large Language Models (LLMs) through Extension of the Common Vulnerability Scoring System (CVSS) Framework

Large Language Models (LLMs) have revolutionized Generative Artificial Intelligence (GenAI) tasks, becoming an integral part of various applications in society, including text generation, translation, summarization, and more. However, their widespread usage emphasizes the critical need to enhance their security posture to ensure the integrity and reliability of their outputs and minimize harmful effects. Prompt injections and training data poisoning attacks are two of the most prominent vulnerabilities in LLMs, which could potentially lead to unpredictable and undesirable behaviors, such as biased outputs, misinformation propagation, and even malicious content generation. The Common Vulnerability Scoring System (CVSS) framework provides a standardized approach to capturing the principal characteristics of vulnerabilities, facilitating a deeper understanding of their severity within the security and AI communities. By extending the current CVSS framework, we generate scores for these vulnerabilities such that organizations can prioritize mitigation efforts, allocate resources effectively, and implement targeted security measures to defend against potential risks.

Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models

Since the 1950s,when the Turing Test was introduced,there has been notable progress in machine language intelligence.Language modeling,crucial for AI development,has evolved from statistical to neural models over the last two decades.Recently,transformer-based Pre-trained Language Models(PLM)have excelled in Natural Language Processing(NLP)tasks by leveraging large-scale training corpora.Increasing the scale of these models enhances performance significantly,introducing abilities like context learning that smaller models lack.The advancement in Large Language Models,exemplified by the development of ChatGPT,has made significant impacts both academically and industrially,capturing widespread societal interest.This survey provides an overview of the development and prospects from Large Language Models(LLM)to Large Multimodal Models(LMM).It first discusses the contributions and technological advancements of LLMs in the field of natural language processing,especially in text generation and language understanding.Then,it turns to the discussion of LMMs,which integrates various data modalities such as text,images,and sound,demonstrating advanced capabilities in understanding and generating cross-modal content,paving new pathways for the adaptability and flexibility of AI systems.Finally,the survey highlights the prospects of LMMs in terms of technological development and application potential,while also pointing out challenges in data integration,cross-modal understanding accuracy,providing a comprehensive perspective on the latest developments in this field.

DeBERTa-GRU: Sentiment Analysis for Large Language Model

Modern technological advancements have made social media an essential component of daily life.Social media allow individuals to share thoughts,emotions,and ideas.Sentiment analysis plays the function of evaluating whether the sentiment of the text is positive,negative,neutral,or any other personal emotion to understand the sentiment context of the text.Sentiment analysis is essential in business and society because it impacts strategic decision-making.Sentiment analysis involves challenges due to lexical variation,an unlabeled dataset,and text distance correlations.The execution time increases due to the sequential processing of the sequence models.However,the calculation times for the Transformer models are reduced because of the parallel processing.This study uses a hybrid deep learning strategy to combine the strengths of the Transformer and Sequence models while ignoring their limitations.In particular,the proposed model integrates the Decoding-enhanced with Bidirectional Encoder Representations from Transformers(BERT)attention(DeBERTa)and the Gated Recurrent Unit(GRU)for sentiment analysis.Using the Decoding-enhanced BERT technique,the words are mapped into a compact,semantic word embedding space,and the Gated Recurrent Unit model can capture the distance contextual semantics correctly.The proposed hybrid model achieves F1-scores of 97%on the Twitter Large Language Model(LLM)dataset,which is much higher than the performance of new techniques.

Enhancing Relational Triple Extraction in Specific Domains:Semantic Enhancement and Synergy of Large Language Models and Small Pre-Trained Language Models

In the process of constructing domain-specific knowledge graphs,the task of relational triple extraction plays a critical role in transforming unstructured text into structured information.Existing relational triple extraction models facemultiple challenges when processing domain-specific data,including insufficient utilization of semantic interaction information between entities and relations,difficulties in handling challenging samples,and the scarcity of domain-specific datasets.To address these issues,our study introduces three innovative components:Relation semantic enhancement,data augmentation,and a voting strategy,all designed to significantly improve the model’s performance in tackling domain-specific relational triple extraction tasks.We first propose an innovative attention interaction module.This method significantly enhances the semantic interaction capabilities between entities and relations by integrating semantic information fromrelation labels.Second,we propose a voting strategy that effectively combines the strengths of large languagemodels(LLMs)and fine-tuned small pre-trained language models(SLMs)to reevaluate challenging samples,thereby improving the model’s adaptability in specific domains.Additionally,we explore the use of LLMs for data augmentation,aiming to generate domain-specific datasets to alleviate the scarcity of domain data.Experiments conducted on three domain-specific datasets demonstrate that our model outperforms existing comparative models in several aspects,with F1 scores exceeding the State of the Art models by 2%,1.6%,and 0.6%,respectively,validating the effectiveness and generalizability of our approach.

A large language model-powered literature review for high-angle annular dark field imaging

High-angle annular dark field(HAADF)imaging in scanning transmission electron microscopy(STEM)has become an indispensable tool in materials science due to its ability to offer sub-°A resolution and provide chemical information through Z-contrast.This study leverages large language models(LLMs)to conduct a comprehensive bibliometric analysis of a large amount of HAADF-related literature(more than 41000 papers).By using LLMs,specifically ChatGPT,we were able to extract detailed information on applications,sample preparation methods,instruments used,and study conclusions.The findings highlight the capability of LLMs to provide a new perspective into HAADF imaging,underscoring its increasingly important role in materials science.Moreover,the rich information extracted from these publications can be harnessed to develop AI models that enhance the automation and intelligence of electron microscopes.

LKPNR: Large Language Models and Knowledge Graph for Personalized News Recommendation Framework

Accurately recommending candidate news to users is a basic challenge of personalized news recommendation systems.Traditional methods are usually difficult to learn and acquire complex semantic information in news texts,resulting in unsatisfactory recommendation results.Besides,these traditional methods are more friendly to active users with rich historical behaviors.However,they can not effectively solve the long tail problem of inactive users.To address these issues,this research presents a novel general framework that combines Large Language Models(LLM)and Knowledge Graphs(KG)into traditional methods.To learn the contextual information of news text,we use LLMs’powerful text understanding ability to generate news representations with rich semantic information,and then,the generated news representations are used to enhance the news encoding in traditional methods.In addition,multi-hops relationship of news entities is mined and the structural information of news is encoded using KG,thus alleviating the challenge of long-tail distribution.Experimental results demonstrate that compared with various traditional models,on evaluation indicators such as AUC,MRR,nDCG@5 and nDCG@10,the framework significantly improves the recommendation performance.The successful integration of LLM and KG in our framework has established a feasible way for achieving more accurate personalized news recommendation.Our code is available at https://github.com/Xuan-ZW/LKPNR.

Evaluating the role of large language models in inflammatory bowel disease patient information

This letter evaluates the article by Gravina et al on ChatGPT’s potential in providing medical information for inflammatory bowel disease patients.While promising,it highlights the need for advanced techniques like reasoning+action and retrieval-augmented generation to improve accuracy and reliability.Emphasizing that simple question and answer testing is insufficient,it calls for more nuanced evaluation methods to truly gauge large language models’capabilities in clinical applications.

Potential use of large language models for mitigating students’problematic social media use:ChatGPT as an example

The problematic use of social media has numerous negative impacts on individuals'daily lives,interpersonal relationships,physical and mental health,and more.Currently,there are few methods and tools to alleviate problematic social media,and their potential is yet to be fully realized.Emerging large language models(LLMs)are becoming increasingly popular for providing information and assistance to people and are being applied in many aspects of life.In mitigating problematic social media use,LLMs such as ChatGPT can play a positive role by serving as conversational partners and outlets for users,providing personalized information and resources,monitoring and intervening in problematic social media use,and more.In this process,we should recognize both the enormous potential and endless possibilities of LLMs such as ChatGPT,leveraging their advantages to better address problematic social media use,while also acknowledging the limitations and potential pitfalls of ChatGPT technology,such as errors,limitations in issue resolution,privacy and security concerns,and potential overreliance.When we leverage the advantages of LLMs to address issues in social media usage,we must adopt a cautious and ethical approach,being vigilant of the potential adverse effects that LLMs may have in addressing problematic social media use to better harness technology to serve individuals and society.

Large Language Model Based Semantic Parsing for Intelligent Database Query Engine

With the rapid development of artificial intelligence, large language models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation. These models have great potential to enhance database query systems, enabling more intuitive and semantic query mechanisms. Our model leverages LLM’s deep learning architecture to interpret and process natural language queries and translate them into accurate database queries. The system integrates an LLM-powered semantic parser that translates user input into structured queries that can be understood by the database management system. First, the user query is pre-processed, the text is normalized, and the ambiguity is removed. This is followed by semantic parsing, where the LLM interprets the pre-processed text and identifies key entities and relationships. This is followed by query generation, which converts the parsed information into a structured query format and tailors it to the target database schema. Finally, there is query execution and feedback, where the resulting query is executed on the database and the results are returned to the user. The system also provides feedback mechanisms to improve and optimize future query interpretations. By using advanced LLMs for model implementation and fine-tuning on diverse datasets, the experimental results show that the proposed method significantly improves the accuracy and usability of database queries, making data retrieval easy for users without specialized knowledge.

The Auxiliary Role of Large Language Models in Clinical Dialogues

In recent years,large language models(LLMs)have made significant progress in natural language processing(NLP).These models not only perform well in a variety of language tasks but also show great potential in the medical field.This paper aims to explore the application of LLMs in clinical dialogues,analyzing their role in improving the efficiency of doctor-patient communication,aiding in diagnosis and treatment,and providing emotional support.The paper also discusses the challenges and limitations of the model in terms of privacy protection,ethical issues,and practical applications.Through comprehensive analysis,we conclude that applying LLMs in clinical dialogues is promising.However,it requires careful consideration and caution by practitioners in practice.

基于人工智能LLM技术的虚拟患者系统构建与临床教学应用

目的构建一种基于人工智能大语言模型(large language model,LLM)技术、可用于医学教育的新型虚拟患者(virtual patient,VP)系统,评价该系统在基层医生进修学习全科医学临床思维中的应用效果。方法选取2021年1月至2024年2月在东南大学附属中大医院进修的基层社区医生为研究对象,随机分为试验组和对照组,分别采用基于LLM的VP系统教学、传统教学方法进行授课,通过临床思维理论知识考核、临床思维能力考核、课程满意度调查评估教学效果,并对结果进行相应的统计学分析。结果共纳入124名基层社区医生,其中试验组60例、对照组64例,两组在一般基线资料上差异无统计学意义,具有可比性。课程结束后,试验组临床思维理论知识考核成绩显著高于对照组(83.83±3.15 vs.79.92±4.52,P<0.01),且不及格率显著低于对照组(0.00%vs.9.38%,P<0.05);试验组在临床思维能力3个维度(批判性、系统性、循证思维)方面教学后分数均显著高于教学前,而对照组仅在批判性思维维度上教学前后差异有统计学意义;教学后试验组在系统思维、循证思维方面分数均显著高于对照组(P<0.05),但在批判性思维上两组分数差异无统计学意义。试验组对授课的总体满意度也显著高于对照组(93.33%vs.85.48%,P<0.05)。结论基于LLM的VP系统提升了学员对临床思维理论知识的掌握程度,也促进了其临床思维能力的培养,该教学方法可为其他医学教育群体提供新的教学工具和思路。

基于LLM的多粒度口令分析研究

基于口令的认证是常见的身份认证机制。然而,大规模口令泄露事件时有发生,表明口令仍面临着被猜测或者盗用等风险。由于口令可以被视作一种特殊的自然语言,近年来运用自然语言处理技术进行口令分析的研究工作逐渐展开。目前少有工作在大语言模型(LLM,large language model)上探究口令文本分词粒度对口令分析效果的影响。为此,提出了基于LLM的多粒度口令分析框架,总体上沿用预训练范式,在大量未标记数据集上自主学习口令分布先验知识。该框架由同步网络、主干网络、尾部网络3个模块构成。其中,同步网络模块实现了char-level、template-level和chunk-level这3种粒度的口令分词,并提取了口令的字符分布、结构、词块组成等特征知识;主干网络模块构建了通用的口令模型来学习口令组成规律;尾部网络模块生成了候选口令对目标库进行猜测分析。在Tianya、Twitter等8个口令库上进行大量实验,分析总结了多粒度分词下所提框架在不同语言环境中的口令分析效果。实验结果表明,在中文用户场景中,基于char-level和chunk-level分词的框架口令分析性能接近一致,且显著优于基于template-level分词的框架;在英文用户场景中,基于chunk-level分词的框架口令分析性能最佳。

Research status and application of artificial intelligence large models in the oil and gas industry

This article elucidates the concept of large model technology,summarizes the research status of large model technology both domestically and internationally,provides an overview of the application status of large models in vertical industries,outlines the challenges and issues confronted in applying large models in the oil and gas sector,and offers prospects for the application of large models in the oil and gas industry.The existing large models can be briefly divided into three categories:large language models,visual large models,and multimodal large models.The application of large models in the oil and gas industry is still in its infancy.Based on open-source large language models,some oil and gas enterprises have released large language model products using methods like fine-tuning and retrieval augmented generation.Scholars have attempted to develop scenario-specific models for oil and gas operations by using visual/multimodal foundation models.A few researchers have constructed pre-trained foundation models for seismic data processing and interpretation,as well as core analysis.The application of large models in the oil and gas industry faces challenges such as current data quantity and quality being difficult to support the training of large models,high research and development costs,and poor algorithm autonomy and control.The application of large models should be guided by the needs of oil and gas business,taking the application of large models as an opportunity to improve data lifecycle management,enhance data governance capabilities,promote the construction of computing power,strengthen the construction of“artificial intelligence+energy”composite teams,and boost the autonomy and control of large model technology.

Preventing the Immense Increase in the Life-Cycle Energy and Carbon Footprints of LLM-Powered Intelligent Chatbots

Intelligent chatbots powered by large language models(LLMs)have recently been sweeping the world,with potential for a wide variety of industrial applications.Global frontier technology companies are feverishly participating in LLM-powered chatbot design and development,providing several alternatives beyond the famous ChatGPT.However,training,fine-tuning,and updating such intelligent chatbots consume substantial amounts of electricity,resulting in significant carbon emissions.The research and development of all intelligent LLMs and software,hardware manufacturing(e.g.,graphics processing units and supercomputers),related data/operations management,and material recycling supporting chatbot services are associated with carbon emissions to varying extents.Attention should therefore be paid to the entire life-cycle energy and carbon footprints of LLM-powered intelligent chatbots in both the present and future in order to mitigate their climate change impact.In this work,we clarify and highlight the energy consumption and carbon emission implications of eight main phases throughout the life cycle of the development of such intelligent chatbots.Based on a life-cycle and interaction analysis of these phases,we propose a system-level solution with three strategic pathways to optimize the management of this industry and mitigate the related footprints.While anticipating the enormous potential of this advanced technology and its products,we make an appeal for a rethinking of the mitigation pathways and strategies of the life-cycle energy usage and carbon emissions of the LLM-powered intelligent chatbot industry and a reshaping of their energy and environmental implications at this early stage of development.

Enhancing Orthopedic Knowledge Assessments:The Performance of Specialized Generative Language Model Optimization

Objective This study aimed to evaluate and compare the effectiveness of knowledge base-optimized and unoptimized large language models(LLMs)in the field of orthopedics to explore optimization strategies for the application of LLMs in specific fields.Methods This research constructed a specialized knowledge base using clinical guidelines from the American Academy of Orthopaedic Surgeons(AAOS)and authoritative orthopedic publications.A total of 30 orthopedic-related questions covering aspects such as anatomical knowledge,disease diagnosis,fracture classification,treatment options,and surgical techniques were input into both the knowledge base-optimized and unoptimized versions of the GPT-4,ChatGLM,and Spark LLM,with their generated responses recorded.The overall quality,accuracy,and comprehensiveness of these responses were evaluated by 3 experienced orthopedic surgeons.Results Compared with their unoptimized LLMs,the optimized version of GPT-4 showed improvements of 15.3%in overall quality,12.5%in accuracy,and 12.8%in comprehensiveness;ChatGLM showed improvements of 24.8%,16.1%,and 19.6%,respectively;and Spark LLM showed improvements of 6.5%,14.5%,and 24.7%,respectively.Conclusion The optimization of knowledge bases significantly enhances the quality,accuracy,and comprehensiveness of the responses provided by the 3 models in the orthopedic field.Therefore,knowledge base optimization is an effective method for improving the performance of LLMs in specific fields.

Unlocking the Potential:A Comprehensive Systematic Review of ChatGPT in Natural Language Processing Tasks

As Natural Language Processing(NLP)continues to advance,driven by the emergence of sophisticated large language models such as ChatGPT,there has been a notable growth in research activity.This rapid uptake reflects increasing interest in the field and induces critical inquiries into ChatGPT’s applicability in the NLP domain.This review paper systematically investigates the role of ChatGPT in diverse NLP tasks,including information extraction,Name Entity Recognition(NER),event extraction,relation extraction,Part of Speech(PoS)tagging,text classification,sentiment analysis,emotion recognition and text annotation.The novelty of this work lies in its comprehensive analysis of the existing literature,addressing a critical gap in understanding ChatGPT’s adaptability,limitations,and optimal application.In this paper,we employed a systematic stepwise approach following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses(PRISMA)framework to direct our search process and seek relevant studies.Our review reveals ChatGPT’s significant potential in enhancing various NLP tasks.Its adaptability in information extraction tasks,sentiment analysis,and text classification showcases its ability to comprehend diverse contexts and extract meaningful details.Additionally,ChatGPT’s flexibility in annotation tasks reducesmanual efforts and accelerates the annotation process,making it a valuable asset in NLP development and research.Furthermore,GPT-4 and prompt engineering emerge as a complementary mechanism,empowering users to guide the model and enhance overall accuracy.Despite its promising potential,challenges persist.The performance of ChatGP Tneeds tobe testedusingmore extensivedatasets anddiversedata structures.Subsequently,its limitations in handling domain-specific language and the need for fine-tuning in specific applications highlight the importance of further investigations to address these issues.

Large language models in health care:Development,applications,and challenges

Recently,the emergence of ChatGPT,an artificial intelligence chatbot developed by OpenAI,has attracted significant attention due to its exceptional language comprehension and content generation capabilities,highlighting the immense potential of large language models(LLMs).LLMs have become a burgeoning hotspot across many fields,including health care.Within health care,LLMs may be classified into LLMs for the biomedical domain and LLMs for the clinical domain based on the corpora used for pre-training.In the last 3 years,these domain-specific LLMs have demonstrated exceptional perform-ance on multiple natural language processing tasks,surpassing the perform-ance of general LLMs as well.This not only emphasizes the significance of developing dedicated LLMs for the specific domains,but also raises expectations for their applications in health care.We believe that LLMs may be used widely in preconsultation,diagnosis,and management,with appropriate development and supervision.Additionally,LLMs hold tremen-dous promise in assisting with medical education,medical writing and other related applications.Likewise,health care systems must recognize and address the challenges posed by LLMs.

基于大语言模型(LLM)的宝石知识图谱的构建

宝石学知识的来源有书籍、期刊、课程、市场等,宝石知识点繁多且在存储上处于相对孤立的状态,不利于从业者和研究者检索知识,图谱能够处理知识点之间的复杂关联,是常用的结构化数据无法实现的,构建图谱形式的宝石知识库系统可以方便学习和检索。本文介绍了传统的知识图谱构建方法并指出了其中的难点(成本高、工作量大、技术难、容易出错),提出了使用大语言模型(LLM)来完成知识图谱构建中的一些任务来改善成本和工作量的问题;构思了一种基于LLM的知识图谱构建思路(步骤包括数据清洗、知识获取和知识精炼),构建了一个能够覆盖本科阶段宝石知识的宝石知识图谱,对一些查询场景做了展示,经过内部测试评估证明了新方法的可行性和高效率,并展望了该图谱未来可能的应用方向。

基于IPEX-LLM的本地轻量化课程教学智能辅助系统

提出并实现了一个本地轻量化课程教学智能辅助系统.该系统利用IPEX-LLM(Intel PyTorch extention for large language model)加速库,在计算资源受限的设备上高效部署并运行经过QLoRA(quantum-logic optimized resource allocation)框架微调的大语言模型,并结合增强检索技术,实现了智能问答、智能出题、教学大纲生成、教学演示文档生成等4个主要功能模块的课程灵活定制,在帮助教师提高教学备课和授课的质量与效率、保护数据隐私的同时,支撑学生个性化学习并提供实时反馈.在性能实验中,以集成优化后的Chatglm3-6B模型为例,该系统处理64-token输出任务时仅需4.08 s,验证了其在资源受限环境下快速推理的能力.在实践案例分析中,通过与原生Chatgml-6B和ChatGPT4.0在功能实现上的对比,进一步表明了该系统具备优越的准确性和实用性.