Evaluating Privacy Leakage and Memorization Attacks on Large Language Models (LLMs) in Generative AI Applications

The recent interest in the deployment of Generative AI applications that use large language models (LLMs) has brought to the forefront significant privacy concerns, notably the leakage of Personally Identifiable Information (PII) and other confidential or protected information that may have been memorized during training, specifically during a fine-tuning or customization process. We describe different black-box attacks from potential adversaries and study their impact on the amount and type of information that may be recovered from commonly used and deployed LLMs. Our research investigates the relationship between PII leakage, memorization, and factors such as model size, architecture, and the nature of attacks employed. The study utilizes two broad categories of attacks: PII leakage-focused attacks (auto-completion and extraction attacks) and memorization-focused attacks (various membership inference attacks). The findings from these investigations are quantified using an array of evaluative metrics, providing a detailed understanding of LLM vulnerabilities and the effectiveness of different attacks.

GUARDIAN: A Multi-Tiered Defense Architecture for Thwarting Prompt Injection Attacks on LLMs

This paper introduces a novel multi-tiered defense architecture to protect language models from adversarial prompt attacks. We construct adversarial prompts using strategies like role emulation and manipulative assistance to simulate real threats. We introduce a comprehensive, multi-tiered defense framework named GUARDIAN (Guardrails for Upholding Ethics in Language Models) comprising a system prompt filter, pre-processing filter leveraging a toxic classifier and ethical prompt generator, and pre-display filter using the model itself for output screening. Extensive testing on Meta’s Llama-2 model demonstrates the capability to block 100% of attack prompts. The approach also auto-suggests safer prompt alternatives, thereby bolstering language model security. Quantitatively evaluated defense layers and an ethical substitution mechanism represent key innovations to counter sophisticated attacks. The integrated methodology not only fortifies smaller LLMs against emerging cyber threats but also guides the broader application of LLMs in a secure and ethical manner.

A Novel Shilling Attack Detection Model Based on Particle Filter and Gravitation

With the rapid development of e-commerce, the security issues of collaborative filtering recommender systems have been widely investigated. Malicious users can benefit from injecting a great quantities of fake profiles into recommender systems to manipulate recommendation results. As one of the most important attack methods in recommender systems, the shilling attack has been paid considerable attention, especially to its model and the way to detect it. Among them, the loose version of Group Shilling Attack Generation Algorithm (GSAGenl) has outstanding performance. It can be immune to some PCC (Pearson Correlation Coefficient)-based detectors due to the nature of anti-Pearson correlation. In order to overcome the vulnerabilities caused by GSAGenl, a gravitation-based detection model (GBDM) is presented, integrated with a sophisticated gravitational detector and a decider. And meanwhile two new basic attributes and a particle filter algorithm are used for tracking prediction. And then, whether an attack occurs can be judged according to the law of universal gravitation in decision-making. The detection performances of GBDM, HHT-SVM, UnRAP, AP-UnRAP Semi-SAD,SVM-TIA and PCA-P are compared and evaluated. And simulation results show the effectiveness and availability of GBDM.

Threat Modeling-Oriented Attack Path Evaluating Algorithm

In order to evaluate all attack paths in a threat tree,based on threat modeling theory,a weight distribution algorithm of the root node in a threat tree is designed,which computes threat coefficients of leaf nodes in two ways including threat occurring possibility and the degree of damage.Besides,an algorithm of searching attack path was also obtained in accordence with its definition.Finally,an attack path evaluation system was implemented which can output the threat coefficients of the leaf nodes in a target threat tree,the weight distribution information,and the attack paths.An example threat tree is given to verify the effectiveness of the algorithms.

Study on Anti-ship Missile Saturation Attack Model

Based on the analysis for the interception process of ship-to-air missile system to the anti-ship missile stream, the antagonism of ship-to-air missile and anti-ship missile stream was modeled by Monte Carlo method. This model containing the probability of acquiring anti-ship missile, threat estimation, firepower distribution, interception, effectiveness evaluation and firepower turning, can dynamically simulate the antagonism process of anti-ship missile attack stream and anti-air missile weapon system. The anti-ship missile's saturation attack stream for different ship-to-air missile systems can be calculated quantitatively. The simulated results reveal the relations among the anti-ship missile saturation attack and the attack intensity of anti-ship missile, interception mode and the main parameters of anti-air missile weapon system. It provides a theoretical basis for the effective operation of anti-ship missile.

A Novel Attack on Complex APUFs Using the Evolutionary Deep Convolutional Neural Network

As the internet of things(IoT)continues to expand rapidly,the significance of its security concerns has grown in recent years.To address these concerns,physical unclonable functions(PUFs)have emerged as valuable tools for enhancing IoT security.PUFs leverage the inherent randomness found in the embedded hardware of IoT devices.However,it has been shown that some PUFs can be modeled by attackers using machine-learning-based approaches.In this paper,a new deep learning(DL)-based modeling attack is introduced to break the resistance of complex XAPUFs.Because training DL models is a problem that falls under the category of NP-hard problems,there has been a significant increase in the use of meta-heuristics(MH)to optimize DL parameters.Nevertheless,it is widely recognized that finding the right balance between exploration and exploitation when dealing with complex problems can pose a significant challenge.To address these chal-lenges,a novel migration-based multi-parent genetic algorithm(MBMPGA)is developed to train the deep convolutional neural network(DCNN)in order to achieve a higher rate of accuracy and convergence speed while decreas-ing the run-time of the attack.In the proposed MBMPGA,a non-linear migration model of the biogeography-based optimization(BBO)is utilized to enhance the exploitation ability of GA.A new multi-parent crossover is then introduced to enhance the exploration ability of GA.The behavior of the proposed MBMPGA is examined on two real-world optimization problems.In benchmark problems,MBMPGA outperforms other MH algorithms in convergence rate.The proposed model are also compared with previous attacking models on several simulated challenge-response pairs(CRPs).The simulation results on the XAPUF datasets show that the introduced attack in this paper obtains more than 99%modeling accuracy even on 8-XAPUF.In addition,the proposed MBMPGA-DCNN outperforms the state-of-the-art modeling attacks in a reduced timeframe and with a smaller number of required sets of CRPs.The area under the curve(AUC)of MBMPGA-DCNN outperforms other architectures.MBMPGA-DCNN achieved sensitivities,specificities,and accuracies of 99.12%,95.14%,and 98.21%,respectively,in the test datasets,establishing it as the most successful method.

An Attack Modeling Based on Colored Petri Net

A color petri net (CPN) based attack modeling approach is addressed. Compared with graph-based modeling, CPN based attack model is flexible enough to model Internet intrusions, because of their static and dynamic features. The processes and rules of building CPN based attack model from attack tree are also presented. In order to evaluate the risk of intrusion, some cost elements are added to CPN based attack modeling. This extended model is useful in intrusion detection and risk evaluation. Experiences show that it is easy to exploit CPN based attack modeling approach to provide the controlling functions, such as intrusion response and intrusion defense. A case study given in this paper shows that CPN based attack model has many unique characters which attack tree model hasn’t.

Algebraic Attack on Filter-Combiner Model Keystream Generators

Algebraic attack was applied to attack Filter-Combintr model keystreamgenerators. We proposed the technique of function composition to improve the model, and the improvedmodel can resist the algebraic attack. A new criterion for designing Filter-Combiner model was alsoproposed: the total length I. of Linear Finite State Machines used in the model should be largeenough and the degree d of Filter-Combiner function should be approximate [L/2].

基于Stacking融合模型的Web攻击检测方法

随着计算机技术与互联网技术的飞速发展,Web应用在人们的生产与生活中扮演着越来越重要的角色。但是在人们的日常生活与工作中带来了更多便捷的同时,却也带来了严重的安全隐患。在开发Web应用的过程中,大量不规范的新技术应用引入了很多的网站漏洞。攻击者可以利用Web应用开发过程中的漏洞发起攻击,当Web应用受到攻击时会造成严重的数据泄露和财产损失等安全问题,因此Web安全问题一直受到学术界和工业界的关注。超文本传输协议(HTTP)是一种在Web应用中广泛使用的应用层协议。随着HTTP协议的大量使用,在HTTP请求数据中包含了大量的实际入侵,针对HTTP请求数据进行Web攻击检测的研究也开始逐渐被研究人员所重视。本文提出了一种基于Stacking融合模型的Web攻击检测方法,针对每一条文本格式的HTTP请求数据,首先进行格式化处理得到既定的格式,结合使用Word2Vec方法和TextCNN模型将其转换成向量化表示形式;然后利用Stacking模型融合方法,将不同的子模型(使用配置不同尺寸过滤器的Text-CNN模型搭配不同的检测算法)进行融合搭建出Web攻击检测模型,与融合之前单独的子模型相比在准确率、召回率、F1值上都有所提升。本文所提出的Web攻击检测模型在公开数据集和真实环境数据上都取得了更加稳定的检测性能。

基于引导扩散模型的自然对抗补丁生成方法

近年来,物理世界中的对抗补丁攻击因其对深度学习模型安全的影响而引起了广泛关注.现有的工作主要集中在生成在物理世界中攻击性能良好的对抗补丁,没有考虑到对抗补丁图案与自然图像的差别,因此生成的对抗补丁往往不自然且容易被观察者发现.为了解决这个问题,本文提出了一种基于引导的扩散模型的自然对抗补丁生成方法.具体而言,本文通过解析目标检测器的输出构建预测对抗补丁攻击成功率的预测器,利用该预测器的梯度作为条件引导预训练的扩散模型的逆扩散过程,从而生成自然度更高且保持高攻击成功率的对抗补丁.本文在数字世界和物理世界中进行了广泛的实验,评估了对抗补丁针对各种目标检测模型的攻击效果以及对抗补丁的自然度.实验结果表明,通过将所构建的攻击成功率预测器与扩散模型相结合,本文的方法能够生成比现有方案更自然的对抗补丁,同时保持攻击性能.

自杀式恐怖袭击防控探究

自杀式恐怖袭击具有恐怖袭击模式与恐怖袭击手段的多重属性,其运行满足以恐怖主体为核心耦合袭击目标、袭击手段、袭击时间和袭击后果等多维恐怖袭击要素的基本存续结构,亦以袭击主体的消亡带动袭击目标受袭形成袭击手段。为有效认知自杀式恐怖袭击以提升恐怖主义治理效能,从袭击主体、袭击工具和袭击行动的消亡性对其进行剖析,发现基于行为体、承袭体和致损体生成致灾因子、构成致灾机理、形成防控困境,同时也回溯治理逻辑。加强对自杀式恐怖袭击的有效研究,有利于丰富恐怖袭击的研究体系、提升恐怖袭击的应对能力,以期进一步完善恐怖主义的治理工作。

基于感知相似性的多目标优化隐蔽图像后门攻击

深度学习模型容易受到后门攻击,在处理干净数据时表现正常,但在处理具有触发模式的有毒样本时会表现出恶意行为.然而,目前大多数后门攻击产生的后门图像容易被人眼察觉,导致后门攻击隐蔽性不足.因此提出了一种基于感知相似性的多目标优化隐蔽图像后门攻击方法.首先,使用感知相似性损失函数减少后门图像与原始图像之间的视觉差异.其次,采用多目标优化方法解决中毒模型上任务间冲突的问题,从而确保模型投毒后性能稳定.最后,采取了两阶段训练方法,使触发模式的生成自动化,提高训练效率.最终实验结果表明,在干净准确率不下降的情况下,人眼很难将生成的后门图像与原始图像区分开.同时,在目标分类模型上成功进行了后门攻击,all-to-one攻击策略下所有实验数据集的攻击成功率均达到了100%.相比其他隐蔽图像后门攻击方法,具有更好的隐蔽性.

基于凝聚层次聚类算法的ATT&CK模型改进

在应用ATT&CK模型(网络攻击模型)进行网络安全威胁分析的过程中,ATT&CK模型提供的技术集合过于复杂。针对ATT&CK模型应用复杂的问题,论文对模型的技术集进行聚类简化研究,提出了基于聚类算法的模型改进方法,首先对ATT&CK模型的技术集合进行量化和聚类趋势评估,然后对量化的数据应用凝聚层次聚类算法得到简化的聚类结果,最后通过实验验证模型改进有效性。

基于深度学习的自然语言处理鲁棒性研究综述

近年来,基于深度神经网络的模型在几乎所有自然语言处理任务上都取得了非常好的效果,在很多任务上甚至超越了人类.展现了极强能力的大规模语言模型也为自然语言处理模型的发展与落地提供了新的机遇和方向.然而,这些在基准测试集合上取得很好结果的模型在实际应用中的效果却经常大打折扣.近期的一些研究还发现,在测试数据上替换一个相似词语、增加一个标点符号,甚至只是修改一个字母都可能使得这些模型的预测结果发生改变,效果大幅度下降.即使是大型语言模型,也会因输入中的微小扰动而改变其预测结果.什么原因导致了这种现象的发生?深度神经网络模型真的如此脆弱吗?如何才能避免这种问题的出现?这些问题近年来受到了越来越多的关注,诸多有影响力的工作都不约而同地从不同方面讨论了自然语言处理的鲁棒性问题.在本文中,我们从自然语言处理任务的典型范式出发,从数据构建、模型表示、对抗攻防以及评估评价等四个方面对自然语言处理鲁棒性相关研究进行了总结和归纳,并对最新进展进行了介绍,最后探讨了未来的可能研究方向以及我们对自然语言处理鲁棒性问题的一些思考.

基于博弈论的弹目攻防决策方法研究

针对空战环境中弹目攻防双方的对抗特性,提出了一种基于博弈论的弹目攻防决策方法。基于导弹目标运动数学关系得到状态方程,根据弹目攻防对抗机理建立“一对一导弹-目标”动态博弈模型,确定弹目双方策略集与收益矩阵,提出混合策略纳什均衡求解方法,并结合模型滚动预测方法获得该策略空间的纳什均衡点。算例仿真结果表明,基于混合策略下导弹制导律为该策略空间的纳什均衡点,且该方法可以减小导弹对目标的脱靶量,提高导弹的命中精度,为导弹攻防作战提供了依据。

受压混凝土中硫酸根离子的扩散与损伤机理分析

针对沿海、盐渍土环境下荷载和硫酸盐侵蚀的耦合作用对混凝土结构耐久性造成的损伤,提出了一种受压混凝土中硫酸根离子扩散的数值模型。首先,基于Fick第二定律,考虑应力和混凝土孔隙率之间的关系建立了荷载作用下混凝土内硫酸根离子的理论扩散模型;其次,通过自编程序构建包含水泥砂浆、界面过渡区和天然骨料的混凝土三相随机凸多边形骨料模型,从而实现了受压混凝土中硫酸根离子扩散的细观模拟;最后,通过与受压混凝土在硫酸盐溶液中全浸泡的实验结果对比分析,验证了理论模型和细观模型的有效性。进而对不同水灰比的受压混凝土试件在不同浓度硫酸盐溶液中的离子扩散与损伤过程进行了数值分析,分析结果表明:随着压应力水平的提高,同深度处硫酸根离子浓度逐渐减小;与压应力相比,侵蚀溶液浓度和水灰比对硫酸根离子扩散的影响更加明显;水灰比对压应力抑制离子扩散程度的影响要大于侵蚀溶液浓度;适当减小水灰比更有利于受压状态下的混凝土抵抗硫酸盐侵蚀。

基于JSMA对抗攻击的去除深度神经网络后门防御方案

深度学习模型缺乏透明性和可解释性,在推理阶段触发恶意攻击者设定的后门时,模型会出现异常行为,导致性能下降。针对此问题,文章提出一种基于JSMA对抗攻击的去除深度神经网络后门防御方案。首先通过模拟JSMA产生的特殊扰动还原潜藏的后门触发器,并以此为基础模拟还原后门触发图案;然后采用热力图定位还原后隐藏触发器的权重位置;最后使用脊回归函数将权重置零,有效去除深度神经网络中的后门。在MNIST和CIFAR10数据集上对模型性能进行测试,并评估去除后门后的模型性能,实验结果表明,文章所提方案能有效去除深度神经网络模型中的后门,而深度神经网络的测试精度仅下降了不到3%。

欺骗防御技术发展及其大语言模型应用探索

欺骗防御作为主动防御中最具发展前景的技术,帮助防御者面对高隐蔽未知威胁化被动为主动,打破攻守间天然存在的不平衡局面.面对潜在的威胁场景,如何利用欺骗防御技术有效地帮助防御者做到预知威胁、感知威胁、诱捕威胁,均为目前需要解决的关键问题.博弈理论与攻击图模型在主动防御策略制定、潜在风险分析等方面提供了有力支撑,总结回顾了近年来二者在欺骗防御中的相关工作.随着大模型技术的快速发展,大模型与网络安全领域的结合也愈加紧密,通过对传统欺骗防御技术的回顾,提出了一种基于大模型的智能化外网蜜点生成技术,实验分析验证了外网蜜点捕获网络威胁的有效性,与传统Web蜜罐相比较,在仿真性、稳定性与灵活性等方面均有所提升.为增强蜜点间协同合作、提升对攻击威胁的探查与感知能力,提出蜜阵的概念.针对如何利用蜜点和蜜阵技术,对构建集威胁预测、威胁感知和威胁诱捕为一体的主动防御机制进行了展望.

基于可学习攻击步长的联合对抗训练方法

对抗训练(AT)是抵御对抗攻击的有力手段。然而,现有方法在训练效率和对抗鲁棒性之间往往难以平衡。部分方法提高训练效率但降低对抗鲁棒性,而其他方法则相反。为了找到最佳平衡点,提出了一种基于可学习攻击步长的联合对抗训练方法(FGSM-LASS)。该方法包括预测模型和目标模型,其中,预测模型为每个样本预测攻击步长,替代FGSM算法的固定大小攻击步长。接着,将目标模型参数和原始样本输入改进的FGSM算法,生成对抗样本。最后,采用联合训练策略,共同训练预测和目标模型。在与最新五种方法比较时,FGSM-LASS在速度上比鲁棒性最优的LAS-AT快6倍,而鲁棒性仅下降1%;与速度相近的ATAS相比,鲁棒性提升3%。实验结果证明,FGSM-LASS在训练速度和对抗鲁棒性之间的权衡表现优于现有方法。

多维动态网络的密钥认证数学建模与仿真

多维动态网络在密钥认证过程中,若加密的密钥安全程度较低,则会被量子计算机解码,使传输的隐私数据出现泄露。为有效提升密钥的安全认证效果,提出量子攻击下多维动态网络密钥认证数学建模方法。方法建立用于多维动态网络的密钥认证模型,在建立的认证模型中,对网络中待传输数据实施冗余清洗,并使用高级加密标准(Advanced Encryption Standard,AES)算法生成密钥对待传输数据初次加密;添加一串噪声数据在加密数据中,并对其二次加密,进一步提升数据的抗量子攻击性能,建立多维动态网络的安全密钥传输模型,完成密钥的安全传输。根据解密方法结合密钥对数据安全解密,完成多维动态网络密钥的安全认证。实验结果表明,利用上述方法开展网络密钥认证时,加解密时间短,认证准确率在85%以上,且抗量子攻击能力强。