车联网中基于联邦深度强化学习的任务卸载算法

Task Offloading Algorithm Based on Federated Deep Reinforcement Learning for Internet ofVehicles

随着车联网应用服务体系日益丰富,计算资源有限的车辆难以处理这些计算密集和时延敏感的车联网应用。计算卸载作为移动边缘计算中的一种关键技术可以解决这一难题。对于车联网中动态的多车辆多路侧单元的任务卸载环境,提出了一种基于联邦深度强化学习的任务卸载算法。该算法将每辆车都看作是智能体,采用联邦学习的框架训练各智能体,各智能体分布式决策卸载方案,以最小化系统的平均响应时间。设置评估实验,在多种动态变化的场景下对提出的算法的性能进行对比分析。实验结果显示,提出的算法求解出的系统平均响应时间短于基于规则的算法和多智能体深度强化学习算法,接近于理想方案,且求解时间远短于理想方案。实验结果表明,所提算法能够在可接受的算法执行时间内求解出接近于理想方案的系统平均响应时间。

With the rapid development of the service system of Internet of Vehicles applications,vehicles with limited computational resources have difficulty in handling these computation-intensive and latency-sensitive applications.As a key technique in mobile edge computing,task offloading can address the challenge.Specially,a task offloading algorithm based on federated deep reinforcement learning(TOFDRL)is proposed for dynamic multi-vehicle multi-road-side-unit(multi-RSU)task offloading environment in Internet of Vehicles.Each vehicle is considered as an agent,and a federated learning framework is used to train each agent.Each agent makes distributed decisions,aiming to minimize the average system response time.Evaluation experiments are set up to compare and analyze the performance of the proposed algorithm under a variety of dynamically changing scenarios.Si-mulation results show that the average response time of system solved by the proposed algorithm is shorter than that of the rule-based algorithm and the multi-agent deep reinforcement learning algorithm,close to the ideal scheme,and its solution time is much shorter than the ideal solution.Experimental results demonstrate that the proposed algorithm is able to solve an average system response time which is close to the ideal solution within an acceptable execution time.