Evaluating Privacy Leakage and Memorization Attacks on Large Language Models (LLMs) in Generative AI Applications

The recent interest in the deployment of Generative AI applications that use large language models (LLMs) has brought to the forefront significant privacy concerns, notably the leakage of Personally Identifiable Information (PII) and other confidential or protected information that may have been memorized during training, specifically during a fine-tuning or customization process. We describe different black-box attacks from potential adversaries and study their impact on the amount and type of information that may be recovered from commonly used and deployed LLMs. Our research investigates the relationship between PII leakage, memorization, and factors such as model size, architecture, and the nature of attacks employed. The study utilizes two broad categories of attacks: PII leakage-focused attacks (auto-completion and extraction attacks) and memorization-focused attacks (various membership inference attacks). The findings from these investigations are quantified using an array of evaluative metrics, providing a detailed understanding of LLM vulnerabilities and the effectiveness of different attacks.

A Novel Shilling Attack Detection Model Based on Particle Filter and Gravitation

With the rapid development of e-commerce, the security issues of collaborative filtering recommender systems have been widely investigated. Malicious users can benefit from injecting a great quantities of fake profiles into recommender systems to manipulate recommendation results. As one of the most important attack methods in recommender systems, the shilling attack has been paid considerable attention, especially to its model and the way to detect it. Among them, the loose version of Group Shilling Attack Generation Algorithm (GSAGenl) has outstanding performance. It can be immune to some PCC (Pearson Correlation Coefficient)-based detectors due to the nature of anti-Pearson correlation. In order to overcome the vulnerabilities caused by GSAGenl, a gravitation-based detection model (GBDM) is presented, integrated with a sophisticated gravitational detector and a decider. And meanwhile two new basic attributes and a particle filter algorithm are used for tracking prediction. And then, whether an attack occurs can be judged according to the law of universal gravitation in decision-making. The detection performances of GBDM, HHT-SVM, UnRAP, AP-UnRAP Semi-SAD,SVM-TIA and PCA-P are compared and evaluated. And simulation results show the effectiveness and availability of GBDM.

Study on Anti-ship Missile Saturation Attack Model

Based on the analysis for the interception process of ship-to-air missile system to the anti-ship missile stream, the antagonism of ship-to-air missile and anti-ship missile stream was modeled by Monte Carlo method. This model containing the probability of acquiring anti-ship missile, threat estimation, firepower distribution, interception, effectiveness evaluation and firepower turning, can dynamically simulate the antagonism process of anti-ship missile attack stream and anti-air missile weapon system. The anti-ship missile's saturation attack stream for different ship-to-air missile systems can be calculated quantitatively. The simulated results reveal the relations among the anti-ship missile saturation attack and the attack intensity of anti-ship missile, interception mode and the main parameters of anti-air missile weapon system. It provides a theoretical basis for the effective operation of anti-ship missile.

An Attack Modeling Based on Colored Petri Net

A color petri net (CPN) based attack modeling approach is addressed. Compared with graph-based modeling, CPN based attack model is flexible enough to model Internet intrusions, because of their static and dynamic features. The processes and rules of building CPN based attack model from attack tree are also presented. In order to evaluate the risk of intrusion, some cost elements are added to CPN based attack modeling. This extended model is useful in intrusion detection and risk evaluation. Experiences show that it is easy to exploit CPN based attack modeling approach to provide the controlling functions, such as intrusion response and intrusion defense. A case study given in this paper shows that CPN based attack model has many unique characters which attack tree model hasn’t.

Algebraic Attack on Filter-Combiner Model Keystream Generators

Algebraic attack was applied to attack Filter-Combintr model keystreamgenerators. We proposed the technique of function composition to improve the model, and the improvedmodel can resist the algebraic attack. A new criterion for designing Filter-Combiner model was alsoproposed: the total length I. of Linear Finite State Machines used in the model should be largeenough and the degree d of Filter-Combiner function should be approximate [L/2].

GUARDIAN: A Multi-Tiered Defense Architecture for Thwarting Prompt Injection Attacks on LLMs

This paper introduces a novel multi-tiered defense architecture to protect language models from adversarial prompt attacks. We construct adversarial prompts using strategies like role emulation and manipulative assistance to simulate real threats. We introduce a comprehensive, multi-tiered defense framework named GUARDIAN (Guardrails for Upholding Ethics in Language Models) comprising a system prompt filter, pre-processing filter leveraging a toxic classifier and ethical prompt generator, and pre-display filter using the model itself for output screening. Extensive testing on Meta’s Llama-2 model demonstrates the capability to block 100% of attack prompts. The approach also auto-suggests safer prompt alternatives, thereby bolstering language model security. Quantitatively evaluated defense layers and an ethical substitution mechanism represent key innovations to counter sophisticated attacks. The integrated methodology not only fortifies smaller LLMs against emerging cyber threats but also guides the broader application of LLMs in a secure and ethical manner.

Protecting LLMs against Privacy Attacks While Preserving Utility

The recent interest in the deployment of Generative AI applications that use large language models (LLMs) has brought to the forefront significant privacy concerns, notably the leakage of Personally Identifiable Information (PII) and other confidential or protected information that may have been memorized during training, specifically during a fine-tuning or customization process. This inadvertent leakage of sensitive information typically occurs when the models are subjected to black-box attacks. To address the growing concerns of safeguarding private and sensitive information while simultaneously preserving its utility, we analyze the performance of Targeted Catastrophic Forgetting (TCF). TCF involves preserving targeted pieces of sensitive information within datasets through an iterative pipeline which significantly reduces the likelihood of such information being leaked or reproduced by the model during black-box attacks, such as the autocompletion attack in our case. The experiments conducted using TCF evidently demonstrate its capability to reduce the extraction of PII while still preserving the context and utility of the target application.

Unsteady aerodynamics modeling for flight dynamics application

In view of engineering application, it is practicable to decompose the aerodynamics into three components： the static aerodynamics, the aerodynamic increment due to steady rotations, and the aerodynamic increment due to unsteady separated and vortical flow. The first and the second components can be presented in conventional forms, while the third is described using a one-order differential equation and a radial-basis-function （RBF） network. For an aircraft configuration, the mathematical models of 6- component aerodynamic coefficients are set up from the wind tunnel test data of pitch, yaw, roll, and coupled yawroll large-amplitude oscillations. The flight dynamics of an aircraft is studied by the bifurcation analysis technique in the case of quasi-steady aerodynamics and unsteady aerodynam- ics, respectively. The results show that： （1） unsteady aerodynamics has no effect upon the existence of trim points, but affects their stability; （2） unsteady aerodynamics has great effects upon the existence, stability, and amplitudes of periodic solutions; and （3） unsteady aerodynamics changes the stable regions of trim points obviously. Furthermore, the dynamic responses of the aircraft to elevator deflections are inspected. It is shown that the unsteady aerodynamics is beneficial to dynamic stability for the present aircraft. Finally, the effects of unsteady aerodynamics on the post-stall maneuverability

Evacuation simulation considering action of guard in artificial attack

To investigate the evacuation behaviors of pedestrians considering the action of guards and to develop an effective evacuation strategy in an artificial attack, an extended floor field model is proposed. In this model, the artificial attacker's assault on pedestrians, the death of pedestrians, and the guard's capture are involved simultaneously. An alternative evacuation strategy which can largely reduce the number of casualties is developed and the effects of several key parameters such as the deterrence radius and capture distance on evacuation dynamics are studied. The results show that congestion near the exit has dual effects. More specifically, the guard can catch all attackers in a short time because the artificial attackers have a more concentrated distribution, but more casualties can occur because it is hard for pedestrians to escape the assault due to congestion. In contrast, when pedestrians have more preference of approaching the guard, although the guard will take more time to capture the attackers resulting from the dispersion of the attackers, the death toll will decrease. One of the reasons is the dispersal of the crowd, and the decrease in congestion is beneficial for escape. The other is that the attackers will be caught before launching the attack on the people who are around the guard, in other words, the guard protects a large number of pedestrians from being killed. Moreover, increasing capture distance of the guard can effectively reduce the casualties and the catch time. As the deterrence radius reflecting the tendency of escaping from the guard for attackers rises, it becomes more difficult for the guard to catch the attackers and more casualties are caused. However, when the deterrence radius reaches a certain level, the number of deaths is reduced because the attackers prefer to stay as far away as possible from the guard rather than occupy a position where they could assault more people.

A Novel Attack on Complex APUFs Using the Evolutionary Deep Convolutional Neural Network

As the internet of things(IoT)continues to expand rapidly,the significance of its security concerns has grown in recent years.To address these concerns,physical unclonable functions(PUFs)have emerged as valuable tools for enhancing IoT security.PUFs leverage the inherent randomness found in the embedded hardware of IoT devices.However,it has been shown that some PUFs can be modeled by attackers using machine-learning-based approaches.In this paper,a new deep learning(DL)-based modeling attack is introduced to break the resistance of complex XAPUFs.Because training DL models is a problem that falls under the category of NP-hard problems,there has been a significant increase in the use of meta-heuristics(MH)to optimize DL parameters.Nevertheless,it is widely recognized that finding the right balance between exploration and exploitation when dealing with complex problems can pose a significant challenge.To address these chal-lenges,a novel migration-based multi-parent genetic algorithm(MBMPGA)is developed to train the deep convolutional neural network(DCNN)in order to achieve a higher rate of accuracy and convergence speed while decreas-ing the run-time of the attack.In the proposed MBMPGA,a non-linear migration model of the biogeography-based optimization(BBO)is utilized to enhance the exploitation ability of GA.A new multi-parent crossover is then introduced to enhance the exploration ability of GA.The behavior of the proposed MBMPGA is examined on two real-world optimization problems.In benchmark problems,MBMPGA outperforms other MH algorithms in convergence rate.The proposed model are also compared with previous attacking models on several simulated challenge-response pairs(CRPs).The simulation results on the XAPUF datasets show that the introduced attack in this paper obtains more than 99%modeling accuracy even on 8-XAPUF.In addition,the proposed MBMPGA-DCNN outperforms the state-of-the-art modeling attacks in a reduced timeframe and with a smaller number of required sets of CRPs.The area under the curve(AUC)of MBMPGA-DCNN outperforms other architectures.MBMPGA-DCNN achieved sensitivities,specificities,and accuracies of 99.12%,95.14%,and 98.21%,respectively,in the test datasets,establishing it as the most successful method.

Cybersecurity: A Stochastic Predictive Model to Determine Overall Network Security Risk Using Markovian Process

There are several security metrics developed to protect the computer networks. In general, common security metrics focus on qualitative and subjective aspects of networks lacking formal statistical models. In the present study, we propose a stochastic model to quantify the risk associated with the overall network using Markovian process in conjunction with Common Vulnerability Scoring System (CVSS) framework. The model we developed uses host access graph to represent the network environment. Utilizing the developed model, one can filter the large amount of information available by making a priority list of vulnerable nodes existing in the network. Once a priority list is prepared, network administrators can make software patch decisions. Gaining in depth understanding of the risk and priority level of each host helps individuals to implement decisions like deployment of security products and to design network topologies.

Attacks and Countermeasures in Social Network Data Publishing

With the increasing prevalence of social networks, more and more social network data are published for many applications, such as social network analysis and data mining. However, this brings privacy problems. For example, adversaries can get sensitive information of some individuals easily with little background knowledge. How to publish social network data for analysis purpose while preserving the privacy of individuals has raised many concerns. Many algorithms have been proposed to address this issue. In this paper, we discuss this privacy problem from two aspects： attack models and countermeasures. We analyse privacy conceres, model the background knowledge that adversary may utilize and review the recently developed attack models. We then survey the state-of-the-art privacy preserving methods in two categories： anonymization methods and differential privacy methods. We also provide research directions in this area.

基于Stacking融合模型的Web攻击检测方法

随着计算机技术与互联网技术的飞速发展,Web应用在人们的生产与生活中扮演着越来越重要的角色。但是在人们的日常生活与工作中带来了更多便捷的同时,却也带来了严重的安全隐患。在开发Web应用的过程中,大量不规范的新技术应用引入了很多的网站漏洞。攻击者可以利用Web应用开发过程中的漏洞发起攻击,当Web应用受到攻击时会造成严重的数据泄露和财产损失等安全问题,因此Web安全问题一直受到学术界和工业界的关注。超文本传输协议(HTTP)是一种在Web应用中广泛使用的应用层协议。随着HTTP协议的大量使用,在HTTP请求数据中包含了大量的实际入侵,针对HTTP请求数据进行Web攻击检测的研究也开始逐渐被研究人员所重视。本文提出了一种基于Stacking融合模型的Web攻击检测方法,针对每一条文本格式的HTTP请求数据,首先进行格式化处理得到既定的格式,结合使用Word2Vec方法和TextCNN模型将其转换成向量化表示形式;然后利用Stacking模型融合方法,将不同的子模型(使用配置不同尺寸过滤器的Text-CNN模型搭配不同的检测算法)进行融合搭建出Web攻击检测模型,与融合之前单独的子模型相比在准确率、召回率、F1值上都有所提升。本文所提出的Web攻击检测模型在公开数据集和真实环境数据上都取得了更加稳定的检测性能。

HASN:A Hierarchical Attack Surface Network for System Security Analysis

Attack surfaces, as one of the security models, can help people to analyse the security of systems in cyberspace, such as risk assessment by utilizing various security metrics or providing a cost-effective network hardening solution. Numerous attack surface models have been proposed in the past decade,but they are not appropriate for describing complex systems with heterogeneous components. To address this limitation, we propose to use a two-layer Hierarchical Attack Surface Network(HASN) that models the data interactions and resource distribution of the system in a component-oriented view. First, we formally define the HASN by extending the entry point and exit point framework. Second, in order to assess data input risk and output risk on the HASN, we propose two behaviour models and two simulation-based risk metrics. Last, we conduct experiments for three network systems. Our experimental results show that the proposed approach is applicable and effective.

Cybersecurity: A Statistical Predictive Model for the Expected Path Length

The object of this study is to propose a statistical model for predicting the Expected Path Length (expected number of steps the attacker will take, starting from the initial state to compromise the security goal—EPL) in a cyber-attack. The model we developed is based on utilizing vulnerability information along with having host centric attack graph. Utilizing the developed model, one can identify the interaction among the vulnerabilities and individual variables (risk factors) that drive the Expected Path Length. Gaining a better understanding of the relationship between vulnerabilities and their interactions can provide security administrators a better view and an understanding of their security status. In addition, we have also ranked the attributable variables and their contribution in estimating the subject length. Thus, one can utilize the ranking process to take precautions and actions to minimize Expected Path Length.

Constructing Certificateless Encryption with Keyword Search against Outside and Inside Keyword Guessing Attacks

Searchable public key encryption is a useful cryptographic paradigm that enables an untrustworthy server to retrieve the encrypted data without revealing the contents of the data. It offers a promising solution to encrypted data retrieval in cryptographic cloud storage. Certificateless public key cryptography (CLPKC) is a novel cryptographic primitive that has many merits. It overcomes the key escrow problem in identity-based cryptography (IBC) and the cumbersome certificate problem in conventional public key cryptography (PKC). Motivated by the appealing features of CLPKC, several certificateless encryption with keyword search (CLEKS) schemes have been presented in the literature. But, our cryptanalysis demonstrates that the previously proposed CLEKS frameworks suffer from the security vulnerability caused by the keyword guessing attack. To remedy the security weakness in the previous frameworks and provide resistance against both inside and outside keyword guessing attacks, we propose a new CLEKS framework. Under the new framework, we design a concrete CLEKS scheme and formally prove its security in the random oracle model. Compared with previous two CLEKS schemes, the proposed scheme has better overall performance while offering stronger security guarantee as it withstands the existing known types of keyword guessing attacks.

自杀式恐怖袭击防控探究

自杀式恐怖袭击具有恐怖袭击模式与恐怖袭击手段的多重属性,其运行满足以恐怖主体为核心耦合袭击目标、袭击手段、袭击时间和袭击后果等多维恐怖袭击要素的基本存续结构,亦以袭击主体的消亡带动袭击目标受袭形成袭击手段。为有效认知自杀式恐怖袭击以提升恐怖主义治理效能,从袭击主体、袭击工具和袭击行动的消亡性对其进行剖析,发现基于行为体、承袭体和致损体生成致灾因子、构成致灾机理、形成防控困境,同时也回溯治理逻辑。加强对自杀式恐怖袭击的有效研究,有利于丰富恐怖袭击的研究体系、提升恐怖袭击的应对能力,以期进一步完善恐怖主义的治理工作。

Non-Homogeneous Stochastic Model for Cyber Security Predictions

Any computer system with known vulnerabilities can be presented using attack graphs. An attacker generally has a mission to reach a goal state that he expects to achieve. Expected Path Length (EPL) [1] in the context of an attack graph describes the length or number of steps that the attacker has to take in achieving the goal state. However, EPL varies and it is based on the “state of vulnerabilities” [2] [3] in a given computer system. Any vulnerability throughout its life cycle passes through several stages that we identify as “states of the vulnerability life cycle” [2] [3]. In our previous studies we have developed mathematical models using Markovian theory to estimate the probability of a given vulnerability being in a particular state of its life cycle. There, we have considered a typical model of a computer network system with two computers subject to three vulnerabilities, and developed a method driven by an algorithm to estimate the EPL of this network system as a function of time. This approach is important because it allows us to monitor a computer system during the process of being exploited. Proposed non-homogeneous model in this study estimates the behavior of the EPL as a function of time and therefore act as an index of the risk associated with the network system getting exploited.

基于引导扩散模型的自然对抗补丁生成方法

近年来,物理世界中的对抗补丁攻击因其对深度学习模型安全的影响而引起了广泛关注.现有的工作主要集中在生成在物理世界中攻击性能良好的对抗补丁,没有考虑到对抗补丁图案与自然图像的差别,因此生成的对抗补丁往往不自然且容易被观察者发现.为了解决这个问题,本文提出了一种基于引导的扩散模型的自然对抗补丁生成方法.具体而言,本文通过解析目标检测器的输出构建预测对抗补丁攻击成功率的预测器,利用该预测器的梯度作为条件引导预训练的扩散模型的逆扩散过程,从而生成自然度更高且保持高攻击成功率的对抗补丁.本文在数字世界和物理世界中进行了广泛的实验,评估了对抗补丁针对各种目标检测模型的攻击效果以及对抗补丁的自然度.实验结果表明,通过将所构建的攻击成功率预测器与扩散模型相结合,本文的方法能够生成比现有方案更自然的对抗补丁,同时保持攻击性能.

Cybersecurity Attacks on Academic Data and Personal Information and the Mediating Role of Education and Employment

The cyberspace has simultaneously presented opportunities and challenges alike for personal data security and privacy, as well as the process of research and learning. Moreover, information such as academic data, research data, personal data, proprietary knowledge, complex equipment designs and blueprints for yet to be patented products has all become extremely susceptible to Cybersecurity attacks. This research will investigate factors that affect that may have an influence on perceived ease of use of Cybersecurity, the influence of perceived ease of use on the attitude towards using Cybersecurity, the influence of attitude towards using Cybersecurity on the actual use of Cybersecurity and the influences of job positions on perceived ease of use of Cybersecurity and on the attitude towards using Cybersecurity and on the actual use of Cybersecurity. A model was constructed to investigate eight hypotheses that are related to the investigation. An online questionnaire was constructed to collect data and results showed that hypotheses 1 to 7 influence were significant. However, hypothesis 8 turned out to be insignificant and no influence was found between job positions and the actual use of Cybersecurity.