基于DQN-DDPG的空地协作边缘计算任务卸载与资源分配研究

Task Offloading and Resource Allocation Based on DQN-DDPG for Aerial-Ground Cooperative Mobile Edge Computing

在基础设施有限的地区或紧急救援场景中,无人机辅助的移动边缘计算被认为是一种有效的解决方案,可处理资源受限的智能设备的计算密集型任务和时延敏感性计算任务。考虑到地面基站和多无人机辅助的多用户空地协作移动边缘计算场景,提出一种联合优化用户关联、子信道分配及边缘服务器计算资源的分配方法,以最小化长期平均时延的任务卸载和资源分配方案。首先,根据用户的随机任务生成无人机移动方案,基于不同的卸载决策建立卸载计算模型和本地计算模型。然后,以最小化长期平均时延为优化目标优化问题。最后,结合DQN与DDPG提出一种基于混合深度强化学习DQN-DDPG的任务卸载和资源分配算法(HDCR),解决离散和连续变量之间的问题和混合决策问题。仿真表明,所提算法相较于基于离散决策的DDCR等算法,在减少平均时延方面性能更优。

In areas with limited infrastructure or emergency rescue scenarios,UAV assisted mobile edge computing is considered an effective solution,which can handle computing intensive tasks and delay sensitive computing tasks of resource constrained intelligent devices.Consider-ing the ground base station and multi UAV assisted multi-user air ground cooperative mobile edge computing scenario,a joint optimization method of user association,subchannel allocation and edge server computing resource allocation is proposed to minimize the long-term aver-age delay of task unloading and resource allocation.Firstly,generate a drone movement plan based on the user's random tasks,and establish offloading calculation models and local calculation models based on different offloading decisions.Then,optimize the problem with the objec-tive of minimizing long-term average latency.Finally,combining DQN and DDPG,a task offloading and resource allocation algorithm(HD-CR)based on hybrid deep reinforcement learning DQN-DDPG is proposed to solve the problems between discrete and continuous variables and mixed decision problems.Simulation results show that the proposed algorithm performs better in reducing average latency compared to al-gorithms such as DDCR based on discrete decision-making.