基于DQN的无人驾驶任务卸载策略

DQN-based driverless task offloading policy

无人驾驶汽车由于其有限的电池寿命和计算能力,难以在保证续航的前提下满足一些时延敏感任务或密集任务的处理需求。为解决该问题,在移动边缘计算(mobile edge computing,MEC)的背景下,提出了一种基于深度Q网络(deep Q-network,DQN)的无人驾驶任务卸载策略。首先,定义了一个基于任务优先级的车—边—云协同任务卸载模型,其需要通过联合优化车辆计算能力与任务卸载策略以获取系统最小延迟和能耗。由于该问题是个混合整数非线性规划问题,所以分两步对其进行求解—通过数学推导得出了最优车辆计算能力的解析解,之后在其数值固定条件下,基于DQN算法获得了任务最佳卸载策略。最后,综合SUMO、PyTorch和Python等工具建立了仿真模型,比较了DQN算法和其他三种算法在任务负载、MEC服务器计算能力以及能耗权重系数变化情况下的性能,实验结果验证了所提策略的可行性和优越性。

Due to its limited battery life and computing power,driverless cars are difficult to meet the processing needs of some delay-sensitive tasks or intensive tasks while ensuring battery life.To solve this problem,in the context of MEC,this paper proposed a driverless task offloading policy based on deep DQN.Firstly,this paper defined a“vehicle-edge-cloud”cooperative task offloading model based on task priority,which needed to jointly optimize the computing power of the vehicle and the task offloading policy to obtain the minimum delay and energy consumption of the system.Since the problem was a mixed-integer nonlinear programming problem and was NP-hard,this paper solved it in two steps—the first step obtained the analytical solution for the optimal computation power of vehicle through mathematical derivation,and then,under the fixed numerical value condition,the DQN algorithm obtained the optimal offloading strategy of the task.Finally,this paper established a simulation model by integrating tools such as SUMO,PyTorch and Python.This paper compared the DQN algorithm and the other three algorithms under different task loads,MEC server computation powers and energy consumption weight co-efficients.The experimental results verify the feasibility and superiority of the proposed policy.