面向非独立同分布数据的联邦学习数据增强方案

Data augmentation scheme for federated learning with non-IID data

为了解决联邦学习节点间数据非独立同分布(non-IID)导致的模型精度不理想的问题,提出一种隐私保护的数据增强方案。首先,提出了面向联邦学习的数据增强框架,参与节点在本地生成虚拟样本并在节点间共享,有效缓解了训练过程中数据分布差异导致的模型偏移问题。其次,基于生成式对抗网络和差分隐私技术,设计了隐私保护的样本生成算法,在保证原数据隐私的前提下生成可用的虚拟样本。最后,提出了隐私保护的标签选取算法,保证虚拟样本的标签同样满足差分隐私。仿真结果表明,在多种non-IID数据划分策略下,所提方案均能有效提高模型精度并加快模型收敛,与基准方法相比,所提方案在极端non-IID场景下能取得25%以上的精度提升。

To solve the problem that the model accuracy remains low when the data are not independent and identically distributed(non-IID) across different clients in federated learning, a privacy-preserving data augmentation scheme was proposed. Firstly, a data augmentation framework for federated learning scenarios was designed. All clients generated synthetic samples locally and shared them with each other, which eased the problem of client drift caused by the difference of clients’ data distributions. Secondly, based on generative adversarial network and differential privacy, a private sample generation algorithm was proposed. It helped clients to generate informative samples while preserving the privacy of clients’ local data. Finally, a differentially private label selection algorithm was proposed to ensure the labels of synthetic samples will not leak information. Simulation results demonstrate that under multiple non-IID data partition strategies, the proposed scheme can consistently improve the model accuracy and make the model converge faster. Compared with the benchmark approaches, the proposed scheme can achieve at least 25% accuracy improvement when each client has only one class of samples.