一种面向云边端系统的分层异构联邦学习方法

A Hierarchically Heterogeneous Federated Learning Method for Cloud-Edge-End System

联邦学习(federated learning)通过用上传模型参数的方式取代了数据传输,降低了隐私泄露的风险.然而,将联邦学习应用到云边端框架下时,一方面,由于云边端存在边缘和终端两层分布式框架,对传统的单层联邦学习提出挑战;另一方面,终端节点因资源异构难以训练相同复杂度的模型,无法满足联邦学习客户端统一模型的假设.针对上述第1个问题,从传统的单层联邦学习方法出发,设计了面向云边端分层部署模型的联邦学习方案;针对第2个问题,通过在终端模型插入分支的方式,将大模型拆分为不同复杂度的小模型适配不同客户端资源状态,从而实现异构联邦学习.同时,考虑到终端存在大量无标签数据无法进行有效模型训练的问题,还提出了针对联邦框架的半监督学习方法,实现对无标签数据的有效利用.最终,通过MNIST和FashionMNIST数据集对方法进行了验证.实验结果表明,在有效避免隐私泄露的前提下,相比于其他同构和异构学习方法,所提方法最大可提升22%的模型准确率;在计算、通信、存储等资源开销上均有明显降低.

Federated learning(FL)has gained applause with great advantages by replacing data transmission with uploading model parameters,effectively avoiding privacy leakage.However,when federated learning is applied to the cloud-edge-end system,for one thing,there are two-level distributed frameworks in cloud-edge-end system,i.e.edge and end,challenging the traditional single-layer federated learning;for another thing,the upper complexity bound of end models is not always the same due to resource heterogeneity,which cannot satisfy the assumption of a unified model for clients in the federated learning framework.To solve the first problem above,based on the traditional single-layer federated learning method,we propose a hierarchically federated learning method for the cloud-edge-end system.For the second problem,we split a deep model into a series of sub-models by inserting early exit branches,where sub-models with different computing complexities match different clients’resource statuses,thus accomplishing hierarchically heterogeneous federated learning.Additionally,due to factors like human resources,there is a lot of unlabeled data on end devices,which cannot be used for effective model training.Therefore,we propose a semi-supervised learning method for federated learning to effectively utilize unlabeled data.Finally,the methods of this paper are verified by MNIST and FashionMNIST datasets.Experimental results show that compared with other methods,the method proposed in this paper can improve accuracy by up to 22%under the premise of privacy security,and at the same time,the resources overhead in computing,communication,and storage have been significantly reduced.