面向数据混合分布的联邦自适应交互模型

Federated Adaptive Interaction Model for Mixed Distribution Data

联邦学习作为一种新兴的分布式机器学习方法,保证了物联网(Internet of things,IoT)设备在数据不出本地的前提下,仅通过传递模型参数来共同维护中央服务器模型,从而达到保护数据隐私安全的目的.传统的联邦学习方法常常在基于设备数据独立同分布的场景下进行联合学习.然而,在实际场景中各设备间的数据样本分布存在差异,使得传统联邦学习方法在非独立同分布(non-independent and identically distributed,Non-IID)的场景下效果不佳.面向Non-IID场景下的混合数据分布问题,提出了新型的联邦自适应交互模型(federated adaptive interaction model,FedAIM)框架,该框架可以同时对不同偏置程度的混合数据进行自适应地交互学习.具体来说,首先,通过引入陆地移动距离(earth mover's distance,EMD)对各客户端的数据分布进行偏置程度度量(bias measurement),并设计极偏服务器和非极偏服务器2个模块分别处理不同偏置程度的数据分布.其次,提出了基于信息熵的模型参数交互机制,使得FedAIM可以有效地聚合极偏服务器和非极偏服务器产生的模型参数,从而有效提高模型的准确率和减少服务器之间的交互轮次.经实验表明,FedAIM在Non-IID混合数据分布场景下的MNIST,Fashion-MNIST,CIFAR-10,SVHN,FEMNIST数据集上准确率均优于已有方法.

Federated learning is an emerging distributed machine learning method that enables mobile phones and IoT devices to learn a shared machine learning model with only transferring model parameters to protect private data.However,traditional federated learning models usually assume training data samples are independent and identically distributed(IID)on the local devices which are not feasible in the real-world,due to the data distributions are different in the different local devices.Hence,existing federated learning models cannot achieve satisfied performance for mixed distribution on the Non-IID data.In this paper,we propose a novel federated adaptive interaction model(FedAIM)for mixed distribution data that can jointly learn IID data and Non-IID data at the same time.In FedAIM,earth mover's distance(EMD)to measure the degree of bias for different client users is introduced for the first time.Then,an extremely biased server and a non-extremely biased server are built to separately process client users with different bias degrees.At the last,a new aggregation mechanism based on information entropy is designed to aggregate and interact model parameters to reduce the number of communication rounds among servers.The experimental results show that the FedAIM outperforms state-of-the-art methods on MNIST,CIFAR-10,Fashion-MNIST,SVHN and FEMNIST of real-world image datasets.