基于深度Q网络的多目标任务卸载算法

Multi-objective task offloading algorithm based on deep Q-network

在移动边缘计算(MEC)中,计算资源和电池容量有限的移动设备(MD)可卸载自身计算密集型应用到边缘服务器上执行,这样不仅可以提高MD计算能力,也能降低能耗。然而,不合理的任务卸载决策不但会延长应用完成时间,而且会大量增加能耗,进而降低用户体验。鉴于此,首先分析MD的移动性和任务间的顺序依赖关系,建立动态MEC网络下的以应用完成时间和能源消耗最小为优化目标的多目标任务卸载问题模型;然后,设计求解该问题的马尔可夫决策过程(MDP)模型,包括状态空间、动作空间和奖励函数,并提出基于深度Q网络(DQN)的多目标任务卸载算法(MTOA-DQN),该算法采用一条轨迹作为经验池的最小单元来改进原始的DQN算法。在多种测试场景下,MTOA-DQN的性能在累积奖励和Cost方面均优于三种对比算法(基于分解的多目标进化算法(MOEA/D)、自适应的DAG任务调度算法(ADTS)和原始的DQN算法),验证了该算法的有效性和可靠性。

For the Mobile Device(MD)with limited computing resources and battery capacity in Mobile Edge Computing(MEC),its computing capacity can be enhanced and its energy consumption can be reduced through offloading its own computing-intensive applications to the edge server.However,unreasonable task offloading strategy will bring a bad experience for users since it will increase the application completion time and energy consumption.To overcome above challenge,firstly,a multi-objective task offloading problem model with minimizing the application completion time and energy consumption as optimization targets was built in the dynamic MEC network via analyzing the mobility of the mobile device and the sequential dependencies between tasks.Then,a Markov Decision Process(MDP)model,including state space,action space,and reward function,was designed to solve this problem,and a Multi-Objective Task Offloading Algorithm based on Deep Q-Network(MTOA-DQN)was proposed,which uses a trajectory as the smallest unit of the experience buffer to improve the original DQN.The proposed MTOA-DQN outperforms three comparison algorithms including MultiObjective Evolutionary Algorithm based on Decomposition(MOEA/D),Adaptive DAG(Directed Acyclic Graph)Tasks Scheduling(ADTS)and original DQN in terms of cumulative reward and cost in a number of test scenarios,verifying the effectiveness and reliability of the algorithm.