基于联邦学习的本地化差分隐私机制研究

A Study of Local Differential Privacy Mechanisms Based on Federated Learning

联邦学习与群体学习作为当前热门的分布式机器学习范式,前者能够保护用户数据不被第三方获得的前提下在服务器中实现模型参数共享计算,后者在无中心服务器的前提下利用区块链技术实现所有用户同等地聚合模型参数。但是,通过分析模型训练后的参数,如深度神经网络训练的权值,仍然可能泄露用户的隐私信息。目前,在联邦学习下运用本地化差分隐私(LDP)保护模型参数的方法层出不穷,但皆难以在较小的隐私预算和用户数量下缩小模型测试精度差。针对此问题,该文提出正负分段机制(PNPM),在聚合前对本地模型参数进行扰动。首先,证明了该机制满足严格的差分隐私定义,保证了算法的隐私性;其次分析了该机制能够在较少的用户数量下保证模型的精度,保证了机制的有效性;最后,在3种主流图像分类数据集上与其他最先进的方法在模型准确性、隐私保护方面进行了比较,表现出了较好的性能。

Federated Learning and swarm Learning,as currently popular distributed machine learning paradigms,the former enables shared computation of model parameters in servers while protecting user data from third parties,while the latter uses blockchain technology to aggregate model parameters equally for all users without a central server.However,by analyzing the parameters after model training,such as the weights of deep neural network training,it is still possible to leak the user's private information.At present,there are several methods for protecting model parameters utilizing Local Differential Privacy(LDP)in federated learning,however it is challenging to reduce the gap in model testing accuracy when there is a limited privacy budget and user base.To solve this problem,a Positive and Negative Piecewise Mechanism(PNPM)is proposed,which perturbs the local model parameters before aggregation.First,it is proved that the mechanism satisfies the strict definition of differential privacy and ensures the privacy of the algorithm;Secondly,it is analyzed that the mechanism can ensure the accuracy of the model under a small number of users and ensure the effectiveness of the mechanism;Finally,it is compared with other state-of-the-art methods in terms of model accuracy and privacy protection on three mainstream image classification datasets and shows a better performance.