面向异构网络的联邦优化算法

Federation Optimization Algorithm for Heterogeneous Networks

联邦学习网络中,全局模型的聚合训练常因边缘设备端的统计异构性而存在收敛问题。针对高度异构环境的适应性问题,提出一种面向异构网络的联邦优化算法q-FedDANE。首先,通过在经典联合近似牛顿型方法中引入衰减参数q,调整衰减梯度校正项和近端项的负面影响,有效提高模型对环境异构性的感知能力,并将每轮算法迭代的设备通信轮次降低至一次,显著减少通信成本和训练开销;其次,模型将随机优化器Adam引入服务器端聚合训练,通过自适应的动态学习率来利用全局信息进行目标优化,加快了模型的收敛速度。实验表明,q-FedDANE算法可以更好地适应环境异构和低设备参与的场景,在高度异构的FEMNIST数据集上,该算法最终获得的测试精度约高出FedDANE的58%。

In federal learning networks,aggregation models tend to exhibit convergence issues due to statistical heterogeneity at edge devices.To improve model adaptability in highly heterogeneous environments,this paper proposes q-FedDANE,a federated optimization algorithm for heterogeneous networks.Firstly,based on the federated approximation Newton-type method,attenuation parameter q is introduced into the model,attenuating negative effects of gradient correction term and proximal term so as to improve the model’s ability of perceiving heterogeneity.In addition,the new algorithm flow carryies out only one round of device communication for each update to reduce the communication cost and training overhead.Secondly,the random optimizer Adam is adopted on server side,dynamically yielding an adaptive learning rate to optimize the objective with global information across edge device networks.This turned out to significantly accelerate the convergence rate of the model.