隐私保护的拜占庭鲁棒联邦学习算法

Privacy preserving byzantine robust federated learning algorithm

联邦学习是一种分布式机器学习范式,其中节点的原始训练集不出本地,它们通过共享模型更新来协作训练机器学习模型。当前联邦学习领域中的隐私保护和拜占庭攻击检测研究大都独立展开,现有的拜占庭攻击检测方法不可直接应用于隐私保护环境,不符合联邦学习的实际应用需求。针对上述问题,提出一种可在数据非独立同分布和隐私保护环境下拜占庭鲁棒的联邦学习算法。首先,以差分隐私技术为模型更新(本地模型梯度信息)提供隐私保护;然后,基于节点上传的历史模型更新对节点当前状态进行可信度评估;最后,根据评估结果进行全局模型聚合。仿真实验结果表明,在节点训练集非独立同分布、隐私保护和拜占庭节点比例为20%~80%的联邦学习环境中,所提算法进行拜占庭节点检测的漏检率和误检率均为0%。同时,随着节点数量的增加,拜占庭节点检测的时间开销呈线性增长的趋势。与现有的拜占庭节点检测算法相比,所提算法在节点数据非独立同分布及模型隐私保护情况下可得到更高精度的全局模型。

Federated learning is a distributed machine learning paradigm,in which the original training sets of the nodes do not have to leave the local area and they collaborate to train machine learning models by sharing model updates.Most of the current privacy-preserving and Byzantine attack detection researches in the field of federated learning are carried out independently,and the existing Byzantine attack detection methods cannot be directly applied to the privacy-preserving environment,which does not meet the practical application requirements of federated learning.To address these problems,this paper proposes a federated learning algorithm for Byzantine robustness in a privacy-preserving environment with data non-independent and identically distributed.First,privacy protection is provided for model updates(local model gradient information)by differential privacy techniques;then the credibility is evaluated for the current state of nodes based on historical model updates uploaded by nodes;and finally,global model aggregation is performed based on the evaluation results.Simulation results show that in a federated learning environment with data non-independent and identically distributed,and with the privacy protection and Byzantine node ratio of 20%~80%,the proposed algorithm performs Byzantine node detection with both the miss detection rate and the false detection rate at 0%.Meanwhile,the time overhead of Byzantine node detection tends to linearly increase with the increase in the number of the nodes.Compared with the existing Byzantine node detection algorithms,the proposed algorithm can obtain a global model with a higher accuracy in the case of data being non-independent and identically distributed and model privacy protection.