基于差分隐私保护知识迁移的联邦学习方法

Federated learning method based on differential privacy protection knowledge transfer

联邦学习解决了机器学习的数据孤岛问题,然而,各方的数据集在数据样本空间和特征空间上可能存在较大差异,导致联邦模型的预测精度下降。针对上述问题,提出了一种基于差分隐私保护知识迁移的联邦学习方法。该方法使用边界扩展局部敏感散列计算各方实例之间的相似度,根据相似度对实例进行加权训练,实现基于实例的联邦迁移学习。在此过程中,实例本身无须透露给其他方,防止了隐私的直接泄露。同时,为了减少知识迁移过程的隐私间接泄露,在知识迁移过程中引入差分隐私机制,对需要在各方之间传输的梯度数据进行扰动,实现知识迁移过程的隐私保护。理论分析表明,知识迁移过程满足ε-差分隐私保护。在XGBoost梯度提升树模型上实现了所提方法,实验结果表明,与无知识迁移方法相比,所提方法使联邦模型测试误差平均下降6%以上。

Federated learning solves the data silo problem of machine learning.However,the dataset of each party may have large differences in the instance space and feature space,which led to the degradation of prediction accuracy of the federated model.To address the above problems,this paper proposed a federated learning method based on differential privacy protection knowledge transfer.The method used boundary-expanding locality-sensitive hashing to calculate the similarity between instances of each party,and carried out weighted training of instances according to the similarity to achieve instance-based fede-rated transfer learning.In the above process,each party didn’t need to disclose their instances to other parties,which could prevent the direct leakage of privacy.Meanwhile,to reduce the indirect privacy leakage in the knowledge transfer process,the proposed method introduced differential privacy mechanism to perturb the gradient data transmitted between all parties,so as to achieve privacy protection in the process of knowledge transfer.Theoretical analysis shows that the knowledge transfer process satisfiesε-differential privacy protection.This paper implemented the proposed method based on the XGBoost model.The experimental results show that,compared with the other methods without knowledge transfer,the proposed method reduces the test error of the federated model by more than 6%on average.