Evaluating Privacy Leakage and Memorization Attacks on Large Language Models (LLMs) in Generative AI Applications

The recent interest in the deployment of Generative AI applications that use large language models (LLMs) has brought to the forefront significant privacy concerns, notably the leakage of Personally Identifiable Information (PII) and other confidential or protected information that may have been memorized during training, specifically during a fine-tuning or customization process. We describe different black-box attacks from potential adversaries and study their impact on the amount and type of information that may be recovered from commonly used and deployed LLMs. Our research investigates the relationship between PII leakage, memorization, and factors such as model size, architecture, and the nature of attacks employed. The study utilizes two broad categories of attacks: PII leakage-focused attacks (auto-completion and extraction attacks) and memorization-focused attacks (various membership inference attacks). The findings from these investigations are quantified using an array of evaluative metrics, providing a detailed understanding of LLM vulnerabilities and the effectiveness of different attacks.

User Profile & Attitude Analysis Based on Unstructured Social Media and Online Activity

As social media and online activity continue to pervade all age groups, it serves as a crucial platform for sharing personal experiences and opinions as well as information about attitudes and preferences for certain interests or purchases. This generates a wealth of behavioral data, which, while invaluable to businesses, researchers, policymakers, and the cybersecurity sector, presents significant challenges due to its unstructured nature. Existing tools for analyzing this data often lack the capability to effectively retrieve and process it comprehensively. This paper addresses the need for an advanced analytical tool that ethically and legally collects and analyzes social media data and online activity logs, constructing detailed and structured user profiles. It reviews current solutions, highlights their limitations, and introduces a new approach, the Advanced Social Analyzer (ASAN), that bridges these gaps. The proposed solutions technical aspects, implementation, and evaluation are discussed, with results compared to existing methodologies. The paper concludes by suggesting future research directions to further enhance the utility and effectiveness of social media data analysis.

A Framework for Cybersecurity Alert Distribution and Response Network (ADRIAN)

Security Information and Event Management (SIEM) platforms are critical for organizations to monitor and manage their security operations centers. However, organizations using SIEM platforms have several challenges such as inefficiency of alert management and integration with real-time communication tools. These challenges cause delays and cost penalties for organizations in their efforts to resolve the alerts and potential security breaches. This paper introduces a cybersecurity Alert Distribution and Response Network (Adrian) system. Adrian introduces a novel enhancement to SIEM platforms by integrating SIEM functionalities with real-time collaboration platforms. Adrian leverages the uniquity of mobile applications of collaboration platforms to provide real-time alerts, enabling a two-way communication channel that facilitates immediate response to security incidents and efficient SIEM platform management. To demonstrate Adrian’s capabilities, we have introduced a case-study that integrates Wazuh, a SIEM platform, to Slack, a collaboration platform. The case study demonstrates all the functionalities of Adrian including the real-time alert distribution, alert customization, alert categorization, and enablement of management activities, thereby increasing the responsiveness and efficiency of Adrian’s capabilities. The study concludes with a discussion on the potential expansion of Adrian’s capabilities including the incorporation of artificial intelligence (AI) for enhanced alert prioritization and response automation.

Ad Blockers & Online Privacy: A Comparative Analysis of Privacy Enhancing Technologies (PET)

Online tracking mechanisms employed by internet companies for user profiling and targeted advertising raise major privacy concerns. Despite efforts to defend against these mechanisms, they continue to evolve, rendering many existing defences ineffective. This study performs a large-scale measurement of online tracking mechanisms across a large pool of websites using the OpenWPM (Open Web Privacy Measurement) platform. It systematically evaluates the effectiveness of several ad blockers and underlying Privacy Enhancing Technologies (PET) that are primarily used to mitigate different tracking techniques. By quantifying the strengths and limitations of these tools against modern tracking methods, the findings highlight gaps in existing privacy protections. Actionable recommendations are provided to enhance user privacy defences, guide tool developers and inform policymakers on addressing invasive online tracking practices.

A Comparative Analysis of Cybersecurity Threat Taxonomies for Healthcare Organizations

Information technology is critical in coordinating patient records, smart devices, operations, and critical infrastructure in healthcare organizations, and their constantly changing digital environment, including suppliers, doctors, insurance providers, and regulatory agencies. This dependence on interdependent systems makes this sector vulnerable to various information technology risks. Such threats include common cybersecurity risks such as data breaches and malware attacks, unique problems occurring in healthcare settings such as unauthorized access to patient records, disruptions in services provided at medical facilities, and potential harm caused to patients due to the compromise of medical devices. The threat taxonomies, such as the Open Threat Taxonomy, NIST, or ENISA, are foundational frameworks for grasping and categorizing IT threats. However, these taxonomies were not specifically designed to deal with the complexities of the healthcare industry. The problem arises from the gap between these taxonomies’ general nature and the industry-specific threats and vulnerabilities that affect healthcare organizations. As a result, many healthcare institutions fail to holistically address and eliminate the unique risks related to confidentiality, integrity, and availability of patients’ data as well as critical systems used in healthcare. This paper aims to narrow this gap by carefully assessing these taxonomies to determine the frame-work best suited for addressing the threat environment in the healthcare sector.

Design & Test of an Advanced Web Security Analysis Tool (AWSAT)

Considering the escalating frequency and sophistication of cyber threats targeting web applications, this paper proposes the development of an automated web security analysis tool to address the accessibility gap for non-security professionals. This paper presents the design and implementation of an automated web security analysis tool, AWSAT, aimed at enabling individuals with limited security expertise to effectively assess and mitigate vulnerabilities in web applications. Leveraging advanced scanning techniques, the tool identifies common threats such as Cross-Site Scripting (XSS), SQL Injection, and Cross-Site Request Forgery (CSRF), providing detailed reports with actionable insights. By integrating sample payloads and reference study links, the tool facilitates informed decision-making in enhancing the security posture of web applications. Through its user-friendly interface and robust functionality, the tool aims to democratize web security practices, empowering a wider audience to proactively safeguard against cyber threats.

Sher: A Secure Broker for DevSecOps and CI/CD Workflows

GitHub Actions, a popular CI/CD platform, introduces significant security challenges due to its integration with GitHub’s open ecosystem and its use of flexible workflow configurations. This paper presents Sher, a Python-based tool that enhances the security of GitHub Actions by automating the detection and remediation of security issues in workflows. Self-Hosted Ephemeral Runner, or Sher, acts as a broker between GitHub’s APIs and a customizable, isolated environment, analyzing workflows through a static rules engine and automatically fixing identified issues. By providing a secure, ephemeral runner environment and a dynamic analysis tool, Sher addresses common misconfigurations and vulnerabilities, contributing to the resilience and integrity of DevSecOps practices within software development pipelines.

Security Vulnerability Analyses of Large Language Models (LLMs) through Extension of the Common Vulnerability Scoring System (CVSS) Framework

Large Language Models (LLMs) have revolutionized Generative Artificial Intelligence (GenAI) tasks, becoming an integral part of various applications in society, including text generation, translation, summarization, and more. However, their widespread usage emphasizes the critical need to enhance their security posture to ensure the integrity and reliability of their outputs and minimize harmful effects. Prompt injections and training data poisoning attacks are two of the most prominent vulnerabilities in LLMs, which could potentially lead to unpredictable and undesirable behaviors, such as biased outputs, misinformation propagation, and even malicious content generation. The Common Vulnerability Scoring System (CVSS) framework provides a standardized approach to capturing the principal characteristics of vulnerabilities, facilitating a deeper understanding of their severity within the security and AI communities. By extending the current CVSS framework, we generate scores for these vulnerabilities such that organizations can prioritize mitigation efforts, allocate resources effectively, and implement targeted security measures to defend against potential risks.

Protecting LLMs against Privacy Attacks While Preserving Utility

The recent interest in the deployment of Generative AI applications that use large language models (LLMs) has brought to the forefront significant privacy concerns, notably the leakage of Personally Identifiable Information (PII) and other confidential or protected information that may have been memorized during training, specifically during a fine-tuning or customization process. This inadvertent leakage of sensitive information typically occurs when the models are subjected to black-box attacks. To address the growing concerns of safeguarding private and sensitive information while simultaneously preserving its utility, we analyze the performance of Targeted Catastrophic Forgetting (TCF). TCF involves preserving targeted pieces of sensitive information within datasets through an iterative pipeline which significantly reduces the likelihood of such information being leaked or reproduced by the model during black-box attacks, such as the autocompletion attack in our case. The experiments conducted using TCF evidently demonstrate its capability to reduce the extraction of PII while still preserving the context and utility of the target application.

Smaller & Smarter: Score-Driven Network Chaining of Smaller Language Models

With the continuous evolution and expanding applications of Large Language Models (LLMs), there has been a noticeable surge in the size of the emerging models. It is not solely the growth in model size, primarily measured by the number of parameters, but also the subsequent escalation in computational demands, hardware and software prerequisites for training, all culminating in a substantial financial investment as well. In this paper, we present novel techniques like supervision, parallelization, and scoring functions to get better results out of chains of smaller language models, rather than relying solely on scaling up model size. Firstly, we propose an approach to quantify the performance of a Smaller Language Models (SLM) by introducing a corresponding supervisor model that incrementally corrects the encountered errors. Secondly, we propose an approach to utilize two smaller language models (in a network) performing the same task and retrieving the best relevant output from the two, ensuring peak performance for a specific task. Experimental evaluations establish the quantitative accuracy improvements on financial reasoning and arithmetic calculation tasks from utilizing techniques like supervisor models (in a network of model scenario), threshold scoring and parallel processing over a baseline study.

Whispered Tuning: Data Privacy Preservation in Fine-Tuning LLMs through Differential Privacy

The proliferation of Large Language Models (LLMs) across various sectors underscored the urgency of addressing potential privacy breaches. Vulnerabilities, such as prompt injection attacks and other adversarial tactics, could make these models inadvertently disclose their training data. Such disclosures could compromise personal identifiable information, posing significant privacy risks. In this paper, we proposed a novel multi-faceted approach called Whispered Tuning to address privacy leaks in large language models (LLMs). We integrated a PII redaction model, differential privacy techniques, and an output filter into the LLM fine-tuning process to enhance confidentiality. Additionally, we introduced novel ideas like the Epsilon Dial for adjustable privacy budgeting for differentiated Training Phases per data handler role. Through empirical validation, including attacks on non-private models, we demonstrated the robustness of our proposed solution SecureNLP in safeguarding privacy without compromising utility. This pioneering methodology significantly fortified LLMs against privacy infringements, enabling responsible adoption across sectors.

Robust Detection and Analysis of Smart Contract Vulnerabilities with Large Language Model Agents

Smart contracts on the Ethereum blockchain continue to revolutionize decentralized applications (dApps) by allowing for self-executing agreements. However, bad actors have continuously found ways to exploit smart contracts for personal financial gain, which undermines the integrity of the Ethereum blockchain. This paper proposes a computer program called SADA (Static and Dynamic Analyzer), a novel approach to smart contract vulnerability detection using multiple Large Language Model (LLM) agents to analyze and flag suspicious Solidity code for Ethereum smart contracts. SADA not only improves upon existing vulnerability detection methods but also paves the way for more secure smart contract development practices in the rapidly evolving blockchain ecosystem.

Object Detection Meets LLMs: Model Fusion for Safety and Security

This paper proposes a novel model fusion approach to enhance predictive capabilities of vision and language models by strategically integrating object detection and large language models. We have named this multimodal integration approach as VOLTRON (Vision Object Linguistic Translation for Responsive Observation and Narration). VOLTRON is aimed at improving responses for self-driving vehicles in detecting small objects crossing roads and identifying merged or narrower lanes. The models are fused using a single layer to provide LLaMA2 (Large Language Model Meta AI) with object detection probabilities from YoloV8-n (You Only Look Once) translated into sentences. Experiments using specialized datasets showed accuracy improvements up to 88.16%. We provide a comprehensive exploration of the theoretical aspects that inform our model fusion approach, detailing the fundamental principles upon which it is built. Moreover, we elucidate the intricacies of the methodologies employed for merging these two disparate models, shedding light on the techniques and strategies used.

ERAD: Enhanced Ransomware Attack Defense System for Healthcare Organizations

Digital integration within healthcare systems exacerbates their vulnerability to sophisticated ransomware threats, leading to severe operational disruptions and data breaches. Current defenses are typically categorized into active and passive measures that struggle to achieve comprehensive threat mitigation and often lack real-time response effectiveness. This paper presents an innovative ransomware defense system, ERAD, designed for healthcare environments that apply the MITRE ATT&CK Matrix to coordinate dynamic, stage-specific countermeasures throughout the ransomware attack lifecycle. By systematically identifying and addressing threats based on indicators of compromise (IOCs), the proposed system proactively disrupts the attack chain before serious damage occurs. Validation is provided through a detailed analysis of a system deployment against LockBit 3.0 ransomware, illustrating significant enhancements in mitigating the impact of the attack, reducing the cost of recovery, and strengthening the cybersecurity framework of healthcare organizations, but also applicable to other non-health sectors of the business world.

GUARDIAN: A Multi-Tiered Defense Architecture for Thwarting Prompt Injection Attacks on LLMs

This paper introduces a novel multi-tiered defense architecture to protect language models from adversarial prompt attacks. We construct adversarial prompts using strategies like role emulation and manipulative assistance to simulate real threats. We introduce a comprehensive, multi-tiered defense framework named GUARDIAN (Guardrails for Upholding Ethics in Language Models) comprising a system prompt filter, pre-processing filter leveraging a toxic classifier and ethical prompt generator, and pre-display filter using the model itself for output screening. Extensive testing on Meta’s Llama-2 model demonstrates the capability to block 100% of attack prompts. The approach also auto-suggests safer prompt alternatives, thereby bolstering language model security. Quantitatively evaluated defense layers and an ethical substitution mechanism represent key innovations to counter sophisticated attacks. The integrated methodology not only fortifies smaller LLMs against emerging cyber threats but also guides the broader application of LLMs in a secure and ethical manner.

Side-Channel Attacks & Data Exfiltration Using Wall Outlet USB Power Adapters

The number and creativity of side channel attacks have increased dramatically in recent years. Of particular interest are attacks leveraging power line communication to 1) gather information on power consumption from the victim and 2) exfiltrate data from compromised machines. Attack strategies of this nature on the greater power grid and building infrastructure levels have been shown to be a serious threat. This project further explores this concept of a novel attack vector by creating a new type of penetration testing tool: an USB power adapter capable of remote monitoring of device power consumption and communicating through powerline communications.

Intelligent ETL for Enterprise Software Applications Using Unstructured Data

Enterprise applications utilize relational databases and structured business processes, requiring slow and expensive conversion of inputs and outputs, from business documents such as invoices, purchase orders, and receipts, into known templates and schemas before processing. We propose a new LLM Agent-based intelligent data extraction, transformation, and load (IntelligentETL) pipeline that not only ingests PDFs and detects inputs within it but also addresses the extraction of structured and unstructured data by developing tools that most efficiently and securely deal with respective data types. We study the efficiency of our proposed pipeline and compare it with enterprise solutions that also utilize LLMs. We establish the supremacy in timely and accurate data extraction and transformation capabilities of our approach for analyzing the data from varied sources based on nested and/or interlinked input constraints.

Generative Adversarial Network Based Approach towards Synthetically Generating Insider Threat Scenarios

This research paper explores the use of Generative Adversarial Networks (GANs) to synthetically generate insider threat scenarios. Insider threats pose significant risks to IT infrastructures, requiring effective detection and mitigation strategies. By training GAN models on historical insider threat data, synthetic scenarios resembling real-world incidents can be generated, including various tactics and procedures employed by insiders. The paper discusses the benefits, challenges, and ethical considerations associated with using GAN-generated data. The findings highlight the potential of GANs in enhancing insider threat detection and response capabilities, empowering organizations to fortify their defenses and proactively mitigate risks posed by internal actors.