自适应量化权重用于通信高效联邦学习

Adaptive quantization weights for communication-efficient federated learning

针对联邦学习训练过程中通信资源有限的问题,本文提出了两种联邦学习算法:自适应量化权重算法和权重复用控制算法,前者对权重的位数进行压缩,减少通信过程中传输的比特数,算法在迭代过程中,自适应调整量化因子,不断减少量化误差;后者能阻止不必要的更新上传,从而减少上传的比特数.基于标准检测数据集Mnist和Cifar10,在CNN和MLP网络模型上做了仿真模拟,实验结果表明,与典型的联邦平均算法相比,提出的算法降低了75%以上的通信成本.

Aiming at the problem of limited communication resources in federated learning and training,two federal learning algorithms are proposed in this paper,the adaptive quantification weighting algorithm and the weighting multiplexing control algorithm,the former compression the median of weight,reduces the number of bits in the transmission in the communication process in iterative process,can adjusts adaptive quantization factor,and constantly reduces the quantization error.The latter prevents unnecessary updates from being uploaded,thereby reducing the number of uploaded bits.Based on the standard detection dataset Mnist and Cifar10,the simulation is carried out on CNN and MLP network models.The experimental results show that the proposed algorithm reduces the communication cost by more than 75%compared with the typical federated average algorithm.