基于稀疏正则双层优化的个性化联邦学习

Personalized Federated Learning Based on Sparsity Regularized Bi-level Optimization

个性化联邦学习侧重于为各客户端提供个性化模型,旨在提高对异构数据的处理性能,然而现有的个性化联邦学习算法大多以增加客户端参数量为代价提高个性化模型的性能,使计算变得复杂.为了解决此问题,文中提出基于稀疏正则双层优化的个性化联邦学习算法(Personalized Federated Learning Based on Sparsity Regularized Bi-level Optimization,pFedSRB),在客户端的个性化更新中引入l 1范数稀疏正则化,提升个性化模型的稀疏度,避免不必要的客户端参数更新,降低模型复杂度.将个性化联邦学习建模为双层优化问题,内层优化采用交替方向乘子法,可提高学习速度.在4个联邦学习基准数据集上的实验表明,pFedSRB在异构数据上表现出色,在提高模型性能的同时有效降低训练用时和空间成本.

Personalized federated learning focuses on providing personalized model for each client,aiming to improve the processing performance on statistically heterogeneous data.However,most existing personalized federated learning algorithms enhance the performance of personalized models at the cost of increasing the number of client parameters and making computation more complex.To address this issue,a personalized federated learning algorithm based on sparsity regularized bi-level optimization(pFedSRB)is proposed in this paper.The l 1 norm sparse regularization is introduced into the personalized update of each client to enhance the sparsity of the personalized model,avoid unnecessary parameter updates of clients,and reduce model complexity.The personalized federated learning problem is formulated as a bi-level optimization problem,and the inner-level optimization of pFedSRB is solved by the alternating direction method of multipliers to improve the learning speed.Experiments on four federated learning benchmark datasets demonstrate that pFedSRB performs well on heterogeneous data,effectively improving model performance while reducing the time and memory costs required for training.