基于DRL的联邦学习节点选择方法

Node selection method in federated learning based on deep reinforcement learning

为了应对设备差异化计算能力及非独立同分布数据对联邦学习性能的影响,高效地调度终端设备完成模型聚合,提出了一种基于深度强化学习的设备节点选择方法。该方法考虑异构节点的训练质量和效率,筛选恶意节点,在提升联邦学习模型准确率的同时,优化训练时延。首先,根据联邦学习中模型分布式训练的特点,构建基于深度强化学习的节点选择系统模型。其次,考虑设备训练时延、模型传输时延和准确率等因素,提出面向节点选择的准确率最优化问题模型。然后,将问题模型构建为马尔可夫决策过程,并设计基于分布式近端策略优化的节点选择算法,在每次训练迭代前选择合理的设备集合完成模型聚合。仿真实验表明,所提方法显著提高了联邦学习的准确率和训练速度,且具有良好的收敛性和稳健性。

To cope with the impact of different device computing capabilities and non-independent uniformly distributed data on federated learning performance,and to efficiently schedule terminal devices to complete model aggregation,a method of node selection based on deep reinforcement learning was proposed.It considered training quality and efficiency of heterogeneous terminal devices,and filtrate malicious nodes to guarantee higher model accuracy and shorter training delay of federated learning.Firstly,according to characteristics of model distributed training in federated learning,a node selection system model based on deep reinforcement learning was constructed.Secondly,considering such factors as device training delay,model transmission delay and accuracy,an optimization model of accuracy for node selection was proposed.Finally,the problem model was constructed as a Markov decision process and a node selection algorithm based on distributed proximal strategy optimization was designed to obtain a reasonable set of devices before each training iteration to complete model aggregation.Simulation results demonstrate that the proposed method significantly improves the accuracy and training speed of federated learning,and its convergence and robustness are also well.