基于原型网络的聚类式联邦学习

Prototypical Network-based Clustered Federated Learning

聚类式联邦学习利用数据分布的差异对用户群体进行聚类,实现个性化的联邦模型训练,可以有效解决联邦学习中的特征分布异构问题.本文针对现有方法通信复杂度较高,难以适用于类别分布异构与小数据场景等问题,基于元学习方法中的原型网络模型提出了一种迭代聚类式联邦学习框架.本文从期望最大化算法的角度出发,利用类原型信息构建用户嵌入表示,并基于此提出用于度量用户与聚类簇之间差异性的模型距离.该迭代式框架交替地执行用户聚类与局部更新以优化全局目标函数.由于新的模型距离可以在服务器端被计算,该框架在每一轮通信中有着近似于FedAvg算法的通信代价.相关实验结果表明了本文提出的方法对于聚类式联邦学习问题,特别是在类别分布异构与小样本数据场景的有效性.

Clustered federated learning leverages the differences among data distributions on clients to partition all clients into several clusters for personalized federated training,which can effectively deal with the feature distribution heterogeneity problem in federated learning.In order to tackle the limitations in the existing methods,such as high communication cost and the inapplicablity to heterogeneous label distributions and few-shot data scenarios,this paper proposes an iterative clustered federated learning framework based on the prototype network model in meta-learning.From the perspective of the expectation maximization algorithm,this paper constructs the client embedding representations using the information regarding class prototypes and then proposes a model distance for measuring the dissimilarities between clients and clusters.The iterative framework conducts client clustering and local updating alternately to optimize the global objective function.Since the novel model distance can be calculated on the server side,the framework almost has the same communication cost compared to FedAvg algorithm in each communication round.The related experimental results show the effectiveness of the method proposed in this paper for clustered federated learning problem,especially in the heterogeneous label distributions and few-shot data scenarios.