基于自适应梯度压缩的高效联邦学习通信机制研究

Research on Efficient Federated Learning Communication Mechanism Based on Adaptive Gradient Compression

针对物联网(IoTs)场景下,联邦学习(FL)过程中大量设备节点之间因冗余的梯度交互通信而带来的不可忽视的通信成本问题,该文提出一种阈值自适应的梯度通信压缩机制。首先,引用了一种基于边缘-联邦学习的高效通信(CE-EDFL)机制,其中边缘服务器作为中介设备执行设备端的本地模型聚合,云端执行边缘服务器模型聚合及新参数下发。其次,为进一步降低联邦学习检测时的通信开销,提出一种阈值自适应的梯度压缩机制(ALAG),通过对本地模型梯度参数压缩,减少设备端与边缘服务器之间的冗余通信。实验结果表明,所提算法能够在大规模物联网设备场景下,在保障深度学习任务完成准确率的同时,通过降低梯度交互通信次数,有效地提升了模型整体通信效率。

Considering the non-negligible communication cost problem caused by redundant gradient interactive communication between a large number of device nodes in the Federated Learning(FL) process in the Internet of Things(IoTs) scenario, gradient communication compression mechanism with adaptive threshold is proposed. Firstly, a structure of Communication-Efficient EDge-Federated Learning(CE-EDFL) is used to prevent device-side data privacy leakage. The edge server acts as an intermediary device to perform device-side local model aggregation, and the cloud performs edge server model aggregation and new parameter delivery.Secondly, in order to reduce further the communication overhead during federated learning detection, a threshold Adaptive Lazily Aggregated Gradient(ALAG) is proposed, which reduces the redundant communication between the device end and the edge server by compressing the gradient parameters of the local model. The experimental results show that the proposed algorithm can effectively improve the overall communication efficiency of the model by reducing the number of gradient interactions while ensuring the accuracy of deep learning tasks in the large-scale IoT device scenario.