基于共享数据集和梯度补偿的分层联邦学习框架

A Hierarchical Federated Learning Framework Based on Shared Dataset and Gradient Compensation

联邦学习允许车辆在本地保留数据并进行模型训练,从而更好地保护用户隐私,但车载传感器和行驶路线等条件不同,参与联邦学习的车辆可能具有不同数据分布,从而降低模型泛化能力,增大收敛难度。为了确保实时性,车联网中广泛应用了异步随机梯度下降技术,但梯度延迟问题会导致模型训练不准确。为了解决上述问题,文章提出一种基于共享数据集和梯度补偿的分层联邦学习框架。该框架使用共享数据集和基于Re LU值加权的聚合方法减少模型偏差,并利用梯度函数的泰勒展开近似原始损失函数,对异步随机梯度下降进行梯度补偿。在MNIST和CIFAR-10数据集上的实验结果表明,与Fed AVG、MOON和Hier FAVG算法相比,该方法平均准确率分别提高了13.8%、2.2%和3.5%,时间开销仅为同步随机梯度下降和异步随机梯度下降的1/2。

Federated learning(FL)enables vehicles to locally retain data for model training,enhancing privacy.However,due to variations in conditions such as onboard sensors and driving routes,vehicles participating in FL may exhibit different data distributions,thereby reducing model generalization and increasing convergence challenges.To ensure real-time performance,asynchronous stochastic gradient descent(SGD)techniques widely employes in Internet of vehicle.Nevertheless,the issue of gradient delay can lead to inaccuracies in model training.To address these challenges,this paper proposes a layered FL framework based on shared datasets and gradient compensation.The framework utilized shared datasets and an aggregation method weighted by ReLU values to reduce model bias.Additionally,it employed a Taylor expansion approximation of the original loss function using the gradient function to compensate for asynchronous SGD.Experimental results on the MNIST and CIFAR-10 datasets indicate that compared to FedAVG,MOON,and HierFAVG,the proposed method achieves an average accuracy improvement of 13.8%,2.2%,and 3.5%,respectively.The time cost is only half that of both synchronous SGD and asynchronous SGD.