Byzantine Robust Federated Learning Scheme Based on Backdoor Triggers

Federated learning is widely used to solve the problem of data decentralization and can provide privacy protectionfor data owners. However, since multiple participants are required in federated learning, this allows attackers tocompromise. Byzantine attacks pose great threats to federated learning. Byzantine attackers upload maliciouslycreated local models to the server to affect the prediction performance and training speed of the global model. Todefend against Byzantine attacks, we propose a Byzantine robust federated learning scheme based on backdoortriggers. In our scheme, backdoor triggers are embedded into benign data samples, and then malicious localmodels can be identified by the server according to its validation dataset. Furthermore, we calculate the adjustmentfactors of local models according to the parameters of their final layers, which are used to defend against datapoisoning-based Byzantine attacks. To further enhance the robustness of our scheme, each localmodel is weightedand aggregated according to the number of times it is identified as malicious. Relevant experimental data showthat our scheme is effective against Byzantine attacks in both independent identically distributed (IID) and nonindependentidentically distributed (non-IID) scenarios.

A Gaussian Noise-Based Algorithm for Enhancing Backdoor Attacks

Deep Neural Networks(DNNs)are integral to various aspects of modern life,enhancing work efficiency.Nonethe-less,their susceptibility to diverse attack methods,including backdoor attacks,raises security concerns.We aim to investigate backdoor attack methods for image categorization tasks,to promote the development of DNN towards higher security.Research on backdoor attacks currently faces significant challenges due to the distinct and abnormal data patterns of malicious samples,and the meticulous data screening by developers,hindering practical attack implementation.To overcome these challenges,this study proposes a Gaussian Noise-Targeted Universal Adversarial Perturbation(GN-TUAP)algorithm.This approach restricts the direction of perturbations and normalizes abnormal pixel values,ensuring that perturbations progress as much as possible in a direction perpendicular to the decision hyperplane in linear problems.This limits anomalies within the perturbations improves their visual stealthiness,and makes them more challenging for defense methods to detect.To verify the effectiveness,stealthiness,and robustness of GN-TUAP,we proposed a comprehensive threat model.Based on this model,extensive experiments were conducted using the CIFAR-10,CIFAR-100,GTSRB,and MNIST datasets,comparing our method with existing state-of-the-art attack methods.We also tested our perturbation triggers using various defense methods and further experimented on the robustness of the triggers against noise filtering techniques.The experimental outcomes demonstrate that backdoor attacks leveraging perturbations generated via our algorithm exhibit cross-model attack effectiveness and superior stealthiness.Furthermore,they possess robust anti-detection capabilities and maintain commendable performance when subjected to noise-filtering methods.

Adaptive Backdoor Attack against Deep Neural Networks

In recent years,the number of parameters of deep neural networks(DNNs)has been increasing rapidly.The training of DNNs is typically computation-intensive.As a result,many users leverage cloud computing and outsource their training procedures.Outsourcing computation results in a potential risk called backdoor attack,in which a welltrained DNN would performabnormally on inputs with a certain trigger.Backdoor attacks can also be classified as attacks that exploit fake images.However,most backdoor attacks design a uniformtrigger for all images,which can be easilydetectedand removed.In this paper,we propose a novel adaptivebackdoor attack.We overcome this defect and design a generator to assign a unique trigger for each image depending on its texture.To achieve this goal,we use a texture complexitymetric to create a specialmask for eachimage,which forces the trigger tobe embedded into the rich texture regions.The trigger is distributed in texture regions,which makes it invisible to humans.Besides the stealthiness of triggers,we limit the range of modification of backdoor models to evade detection.Experiments show that our method is efficient in multiple datasets,and traditional detectors cannot reveal the existence of a backdoor.

A backdoor attack against quantum neural networks with limited information

Backdoor attacks are emerging security threats to deep neural networks.In these attacks,adversaries manipulate the network by constructing training samples embedded with backdoor triggers.The backdoored model performs as expected on clean test samples but consistently misclassifies samples containing the backdoor trigger as a specific target label.While quantum neural networks(QNNs)have shown promise in surpassing their classical counterparts in certain machine learning tasks,they are also susceptible to backdoor attacks.However,current attacks on QNNs are constrained by the adversary's understanding of the model structure and specific encoding methods.Given the diversity of encoding methods and model structures in QNNs,the effectiveness of such backdoor attacks remains uncertain.In this paper,we propose an algorithm that leverages dataset-based optimization to initiate backdoor attacks.A malicious adversary can embed backdoor triggers into a QNN model by poisoning only a small portion of the data.The victim QNN maintains high accuracy on clean test samples without the trigger but outputs the target label set by the adversary when predicting samples with the trigger.Furthermore,our proposed attack cannot be easily resisted by existing backdoor detection methods.

An Improved Optimized Model for Invisible Backdoor Attack Creation Using Steganography

The Deep Neural Networks(DNN)training process is widely affected by backdoor attacks.The backdoor attack is excellent at concealing its identity in the DNN by performing well on regular samples and displaying malicious behavior with data poisoning triggers.The state-of-art backdoor attacks mainly follow a certain assumption that the trigger is sample-agnostic and different poisoned samples use the same trigger.To overcome this problem,in this work we are creating a backdoor attack to check their strength to withstand complex defense strategies,and in order to achieve this objective,we are developing an improved Convolutional Neural Network(ICNN)model optimized using a Gradient-based Optimization(GBO)(ICNN-GBO)algorithm.In the ICNN-GBO model,we are injecting the triggers via a steganography and regularization technique.We are generating triggers using a single-pixel,irregular shape,and different sizes.The performance of the proposed methodology is evaluated using different performance metrics such as Attack success rate,stealthiness,pollution index,anomaly index,entropy index,and functionality.When the CNN-GBO model is trained with the poisoned dataset,it will map the malicious code to the target label.The proposed scheme’s effectiveness is verified by the experiments conducted on both the benchmark datasets namely CIDAR-10 andMSCELEB 1M dataset.The results demonstrate that the proposed methodology offers significant defense against the conventional backdoor attack detection frameworks such as STRIP and Neutral cleanse.

IIS Backdoor刺穿防火墙的后门

现在的防火墙真是越来越厉害，一般的后门根本无法刺穿防火墙的拦截。虽然防火墙在一点点的进步但是后门也没落后。俗话说的好．道高一尺魔高一丈，安全是靠整体的．只要有一点点的疏忽也会被攻破．在本文中我就给大家介绍一款刺穿防火墙的后门——iis backdoor。

DLP:towards active defense against backdoor attacks with decoupled learning process

Deep learning models are well known to be susceptible to backdoor attack,where the attacker only needs to provide a tampered dataset on which the triggers are injected.Models trained on the dataset will passively implant the backdoor,and triggers on the input can mislead the models during testing.Our study shows that the model shows different learning behaviors in clean and poisoned subsets during training.Based on this observation,we propose a general training pipeline to defend against backdoor attacks actively.Benign models can be trained from the unreli-able dataset by decoupling the learning process into three stages,i.e.,supervised learning,active unlearning,and active semi-supervised fine-tuning.The effectiveness of our approach has been shown in numerous experiments across various backdoor attacks and datasets.

XMAM:X-raying models with a matrix to reveal backdoor attacks for federated learning

Federated Learning(FL),a burgeoning technology,has received increasing attention due to its privacy protection capability.However,the base algorithm FedAvg is vulnerable when it suffers from so-called backdoor attacks.Former researchers proposed several robust aggregation methods.Unfortunately,due to the hidden characteristic of backdoor attacks,many of these aggregation methods are unable to defend against backdoor attacks.What's more,the attackers recently have proposed some hiding methods that further improve backdoor attacks'stealthiness,making all the existing robust aggregation methods fail.To tackle the threat of backdoor attacks,we propose a new aggregation method,X-raying Models with A Matrix(XMAM),to reveal the malicious local model updates submitted by the backdoor attackers.Since we observe that the output of the Softmax layer exhibits distinguishable patterns between malicious and benign updates,unlike the existing aggregation algorithms,we focus on the Softmax layer's output in which the backdoor attackers are difficult to hide their malicious behavior.Specifically,like medical X-ray examinations,we investigate the collected local model updates by using a matrix as an input to get their Softmax layer's outputs.Then,we preclude updates whose outputs are abnormal by clustering.Without any training dataset in the server,the extensive evaluations show that our XMAM can effectively distinguish malicious local model updates from benign ones.For instance,when other methods fail to defend against the backdoor attacks at no more than 20%malicious clients,our method can tolerate 45%malicious clients in the black-box mode and about 30%in Projected Gradient Descent(PGD)mode.Besides,under adaptive attacks,the results demonstrate that XMAM can still complete the global model training task even when there are 40%malicious clients.Finally,we analyze our method's screening complexity and compare the real screening time with other methods.The results show that XMAM is about 10–10000 times faster than the existing methods.

Red Alarm for Pre-trained Models:Universal Vulnerability to Neuron-level Backdoor Attacks

The pre-training-then-fine-tuning paradigm has been widely used in deep learning.Due to the huge computation cost for pre-training,practitioners usually download pre-trained models from the Internet and fine-tune them on downstream datasets,while the downloaded models may suffer backdoor attacks.Different from previous attacks aiming at a target task,we show that a backdoored pre-trained model can behave maliciously in various downstream tasks without foreknowing task information.Attackers can restrict the output representations(the values of output neurons)of trigger-embedded samples to arbitrary predefined values through additional training,namely neuron-level backdoor attack(NeuBA).Since fine-tuning has little effect on model parameters,the fine-tuned model will retain the backdoor functionality and predict a specific label for the samples embedded with the same trigger.To provoke multiple labels in a specific task,attackers can introduce several triggers with predefined contrastive values.In the experiments of both natural language processing(NLP)and computer vision(CV),we show that NeuBA can well control the predictions for trigger-embedded instances with different trigger designs.Our findings sound a red alarm for the wide use of pre-trained models.Finally,we apply several defense methods to NeuBA and find that model pruning is a promising technique to resist NeuBA by omitting backdoored neurons.

NBA:defensive distillation for backdoor removal via neural behavior alignment

Recently,deep neural networks have been shown to be vulnerable to backdoor attacks.A backdoor is inserted into neural networks via this attack paradigm,thus compromising the integrity of the network.As soon as an attacker presents a trigger during the testing phase,the backdoor in the model is activated,allowing the network to make specific wrong predictions.It is extremely important to defend against backdoor attacks since they are very stealthy and dangerous.In this paper,we propose a novel defense mechanism,Neural Behavioral Alignment(NBA),for backdoor removal.NBA optimizes the distillation process in terms of knowledge form and distillation samples to improve defense performance according to the characteristics of backdoor defense.NBA builds high-level representations of neural behavior within networks in order to facilitate the transfer of knowledge.Additionally,NBA crafts pseudo samples to induce student models exhibit backdoor neural behavior.By aligning the backdoor neural behavior from the student network with the benign neural behavior from the teacher network,NBA enables the proactive removal of backdoors.Extensive experiments show that NBA can effectively defend against six different backdoor attacks and outperform five state-of-the-art defenses.

基于感知相似性的多目标优化隐蔽图像后门攻击

深度学习模型容易受到后门攻击,在处理干净数据时表现正常,但在处理具有触发模式的有毒样本时会表现出恶意行为.然而,目前大多数后门攻击产生的后门图像容易被人眼察觉,导致后门攻击隐蔽性不足.因此提出了一种基于感知相似性的多目标优化隐蔽图像后门攻击方法.首先,使用感知相似性损失函数减少后门图像与原始图像之间的视觉差异.其次,采用多目标优化方法解决中毒模型上任务间冲突的问题,从而确保模型投毒后性能稳定.最后,采取了两阶段训练方法,使触发模式的生成自动化,提高训练效率.最终实验结果表明,在干净准确率不下降的情况下,人眼很难将生成的后门图像与原始图像区分开.同时,在目标分类模型上成功进行了后门攻击,all-to-one攻击策略下所有实验数据集的攻击成功率均达到了100%.相比其他隐蔽图像后门攻击方法,具有更好的隐蔽性.

CheatKD:基于毒性神经元同化的知识蒸馏后门攻击方法

深度学习模型性能不断提升,但参数规模也越来越大,阻碍了其在边缘端设备的部署应用。为了解决这一问题,研究者提出了知识蒸馏(Knowledge Distillation,KD)技术,通过转移大型教师模型的“暗知识”快速生成高性能的小型学生模型,从而实现边缘端设备的轻量部署。然而,在实际场景中,许多教师模型是从公共平台下载的,缺乏必要的安全性审查,对知识蒸馏任务造成威胁。为此,我们首次提出针对特征KD的后门攻击方法CheatKD,其嵌入在教师模型中的后门,可以在KD过程中保留并转移至学生模型中,进而间接地使学生模型中毒。具体地,在训练教师模型的过程中,CheatKD初始化一个随机的触发器,并对其进行迭代优化,以控制教师模型中特定蒸馏层的部分神经元(即毒性神经元)的激活值,使其激活值趋于定值,以此实现毒性神经元同化操作,最终使教师模型中毒并携带后门。同时,该后门可以抵御知识蒸馏的过滤被传递到学生模型中。在4个数据集和6个模型组合的实验上,CheatKD取得了85%以上的平均攻击成功率,且对于多种蒸馏方法都具有较好的攻击泛用性。

人工智能后门防御评估方法及其架构研究

为了应对人工智能系统可能面临的后门攻击风险,研究人员已经开发了一系列后门防御策略。现有防御方法评估标准的多样性,使得跨方法比较成为一大挑战,因此提出了一种人工智能后门防御统一评估框架。该框架旨在为不同层面(包括数据集级别和模型级别等)的防御策略,提供一个公共的评价标准。在数据集级别,主要通过准确率来评估后门检测的有效性;而在模型级别,则主要关注攻击成功率等指标。人工智能后门防御统一评估框架,能够在相同的评价标准下,对比和分析不同后门防御方法的性能。这不仅有助于识别各方法的优势和不足,还能够提出针对性改进建议。结果表明,人工智能后门防御统一评估框架能有效地评估不同防御策略的性能,为进一步提升人工智能系统的安全性提供重要的参考依据。

基于差分隐私与模型聚类的安全联邦学习方案

联邦学习中的模型安全以及客户隐私是亟待解决的重要挑战。为了同时应对这2大挑战,提出了一项基于差分隐私与模型聚类的联邦学习方案,该方案兼顾模型安全与隐私保护。通过在客户更新中引入局部差分隐私扰乱客户上传的参数以保护客户的隐私数据。为保证对加噪模型更新的精准聚类,首次定义余弦梯度作为聚类指标,并根据聚类结果精准定位恶意模型。最后引入全局差分隐私以抵御潜在的后门攻击。通过理论分析得到全局噪声的噪声边界,并证明了本方案引入的噪声总量低于经典模型安全方案所引入的噪声总量。实验结果表明,本方案能够达成在精度、鲁棒以及隐私3方面的预期目标。

基于模型后门的联邦学习水印

高精度联邦学习模型的训练需要消耗大量的用户本地资源,参与训练的用户能够通过私自出售联合训练的模型获得非法收益.为实现联邦学习模型的产权保护,利用深度学习后门技术不影响主任务精度而仅对少量触发集样本造成误分类的特征,构建一种基于模型后门的联邦学习水印(federated learning watermark based on backdoor,FLWB)方案,能够允许各参与训练的用户在其本地模型中分别嵌入私有水印,再通过云端的模型聚合操作将私有后门水印映射到全局模型作为联邦学习的全局水印.之后提出分步训练方法增强各私有后门水印在全局模型的表达效果,使得FLWB方案能够在不影响全局模型精度的前提下容纳各参与用户的私有水印.理论分析证明了FLWB方案的安全性,实验验证分步训练方法能够让全局模型在仅造成1%主任务精度损失的情况下有效容纳参与训练用户的私有水印.最后,采用模型压缩攻击和模型微调攻击对FLWB方案进行攻击测试,其结果表明FLWB方案在模型压缩到30%时仍能保留80%以上的水印,在4种不同的微调攻击下能保留90%以上的水印,具有很好的鲁棒性.

一种基于后门技术的深度强化学习水印框架

深度强化学习(DRL)已经证明了它在各种复杂任务中的有效性,因其出色的性能使其商业化正在急剧加速。生成一个DRL模型需要大量的计算资源和专业知识,使得一个训练有素的DRL模型已经成为人工智能应用程序和产品的核心知识产权。基于对DRL模型的产权保护,防止非法抄袭、未经授权的分发和复制,提出一种后门技术的DRL水印框架DrlWF,并使用一个全新的评价指标水印动作实现比例来衡量水印性能。通过向训练状态中添加水印,并使用带有水印的水印状态训练模型从而实现将水印嵌入至模型中。框架中的水印嵌入操作可以通过将水印嵌入到少量的训练数据中(仅需0.025%的训练数据)和不影响性能的奖励修改来实现。实验结果证明,在标准状态下,DRL模型仍具有良好的性能;在水印状态下,DRL模型性能将急剧下降,不足原有性能的1%,且水印动作执行比例达到了99%。通过急剧下降的性能以及模型对水印状态的动作表现,即可验证模型的所有权。此外,该水印具有良好的鲁棒性,在模型微调和模型压缩下,模型依然能够识别出水印,性能急剧下降且水印动作执行比例依旧达到了99%以上,证明了该DRL水印具有良好的鲁棒性。

电磁频谱人工智能模型的对抗安全威胁综述

电磁频谱在现代社会中扮演着至关重要的角色,是国家战略资源,为通信、导航、科学研究和国防等领域提供关键支持。为应对电磁频谱高效管理与利用中的诸多挑战,人工智能(Artificial Intelligence, AI)技术在物理层中被广泛应用。然而,研究发现AI模型对于数据的依赖导致其在训练和测试阶段容易受到恶意攻击。为推动针对电磁频谱AI模型的攻击与防御相关研究的发展,保障AI模型的安全应用,提升电磁安全能力,对电磁频谱物理层AI模型的对抗攻击方法进行了回顾,包括在训练阶段和测试阶段的攻击原理与方法。从数据、模型以及电磁信号特性的角度回顾了对抗攻击的评测工作。展望了攻击、评测和系统研发三个具有潜力的研究方向,并做出了总结。

文本后门攻击与防御综述

深度神经网络的安全性和鲁棒性是深度学习领域的研究热点.以往工作主要从对抗攻击角度揭示神经网络的脆弱性,即通过构建对抗样本来破坏模型性能并探究如何进行防御.但随着预训练模型的广泛应用,出现了一种针对神经网络尤其是预训练模型的新型攻击方式——后门攻击.后门攻击向神经网络注入隐藏的后门,使其在处理包含触发器(攻击者预先定义的图案或文本等)的带毒样本时会产生攻击者指定的输出.目前文本领域已有大量对抗攻击与防御的研究,但对后门攻击与防御的研究尚不充分,缺乏系统性的综述.全面介绍文本领域后门攻击和防御技术.首先,介绍文本领域后门攻击基本流程,并从不同角度对文本领域后门攻击和防御方法进行分类,介绍代表性工作并分析其优缺点;之后,列举常用数据集以及评价指标,将后门攻击与对抗攻击、数据投毒2种相关安全威胁进行比较;最后,讨论文本领域后门攻击和防御面临的挑战,展望该新兴领域的未来研究方向.

工业场景下联邦学习中基于模型诊断的后门防御方法

联邦学习作为一种能够解决数据孤岛问题、实现数据资源共享的机器学习方法,其特点与工业设备智能化发展的要求相契合。因此,以联邦学习为代表的人工智能技术在工业互联网中的应用越来越广泛。但是,针对联邦学习架构的攻击手段也在不断更新。后门攻击作为攻击手段的代表之一,有着隐蔽性和破坏性强的特点,而传统的防御方案往往无法在联邦学习架构下发挥作用或者对早期攻击防范能力不足。因此,研究适用于联邦学习架构的后门防御方案具有重大意义。文中提出了一种适用于联邦学习架构的后门诊断方案,能够在无数据情况下利用后门模型的形成特点重构后门触发器,实现准确识别并移除后门模型,从而达到全局模型后门防御的目的。此外,还提出了一种新的检测机制实现对早期模型的后门检测,并在此基础上优化了模型判决算法,通过早退联合判决模式实现了准确率与速度的共同提升。

基于对比训练的联邦学习后门防御方法

针对现有联邦学习后门防御方法不能实现对模型已嵌入后门特征的有效清除同时会降低主任务准确率的问题,提出了一种基于对比训练的联邦学习后门防御方法 Contra FL。利用对比训练来破坏后门样本在特征空间中的聚类过程,使联邦学习全局模型分类结果与后门触发器特征无关。具体而言,服务器通过执行触发器生成算法构造生成器池,以还原全局模型训练样本中可能存在的后门触发器;进而,服务器将触发器生成器池下发给各参与方,各参与方将生成的后门触发器添加至本地样本,以实现后门数据增强,最终通过对比训练有效消除后门攻击的负面影响。实验结果表明,Contra FL能够有效防御联邦学习中的多种后门攻击,且效果优于现有防御方法。