云边协同计算中基于深度强化学习的任务二次申请卸载策略

Task secondary application offloading strategy based on deep reinforcement learning in cloud-edge collaborative computing

现有的任务卸载策略通常在一个时隙内制定卸载决策,没有考虑多个卸载时隙间的内在联系,因此无法根据任务的实际需求进行卸载。针对该问题,提出了一种基于深度强化学习的任务二次申请卸载策略(DQN-TSAO)。首先提出了一种支持任务进行二次申请卸载的云边端三层架构,建立了任务卸载优先级模型、时延模型和能耗模型;然后以最小化系统能耗为目标,将能耗优化问题转变为最大累积卸载奖励的马尔可夫决策过程;最后通过DQN-TSAO算法提取各个时隙的任务卸载特征,使任务在与环境不断交互的过程中获得多个时隙内的最佳卸载决策。仿真结果表明DQN-TSAO算法能够有效降低一段时间内的系统总能耗。

Existing task offloading strategy usually makes offloading decision within one time slot without considering the internal relationship between multiple offload time slots,so they cannot be offloaded according to the actual needs of tasks.To solve this problem,this paper proposed a task secondary application offloading strategy based on deep Q network(DQN-TSAO).Firstly,this paper introduced a three-layer of cloud-edge-end architecture that supported task secondary application offloading,and established priority model,delay model and energy consumption model for task offloading.Secondly,aiming at minimizing system energy consumption,it transformed the energy consumption optimization problem into a Markov decision process problem of maximum cumulative offloading reward.Finally,DQN-TSAO algorithm could extract the task offload characteristics of each time slot,which enabled the task to obtain the optimal offloading decision of multiple time slots in the continuous interaction with the environment.Simulation results validate that DQN-TSAO algorithm can effectively reduce the total energy consumption of the system in a period of time.