基于深度强化学习的云边协同计算迁移研究

Deep Reinforcement Learning Based Cloud-Edge Collaborative Computation Offloading Mechanism

基于单一边缘节点计算、存储资源的有限性及大数据场景对高效计算服务的需求,本文提出了一种基于深度强化学习的云边协同计算迁移机制.具体地,基于计算资源、带宽和迁移决策的综合性考量,构建了一个最小化所有用户任务执行延迟与能耗权重和的优化问题.基于该优化问题提出了一个异步云边协同的深度强化学习算法,该算法充分利用了云边双方的计算能力,可有效满足大数据场景对高效计算服务的需求;同时,面向边缘云中边缘节点所处环境的多样及动态变化性,该算法能自适应地调整迁移策略以实现系统总成本的最小化.最后,大量的仿真结果表明本文所提出的算法具有收敛速度快、鲁棒性高等特点,并能够以最低的计算成本获得近似贪心算法的最优迁移决策.

Based on the computation and storage resources limitation of single edge node and the demand for efficient computing services in big data scenarios,this paper proposes a deep reinforcement learning based cloud-edge collaborative computation offloading mechanism.Specifically,based on a comprehensive consideration of computing resources,bandwidth and offloading policy,an optimization problem is formulated to minimize the weight sum of execution delay and energy consumption of all user tasks.An asynchronous cloud-edge collaborative deep reinforcement learning(ACEC-DRL)algorithm is proposed to solve such optimization problem.This algorithm can effectively satisfy the demand of efficient computing services in big data scenario by jointly leveraging the computation capabilities of cloud and edge nodes.Meanwhile,under the various and dynamic environments of edge nodes in the edge cloud,this algorithm can adaptively adjust offloading policy to achieve the minimization of system cost.Finally,the extensive simulation results show that the proposed ACEC-DRL algorithm has the characteristics of fast convergence rate and high robustness,and its optimal offloading policy closely approximates to the solution of greedy algorithm with the lowest computation cost.