基于深度强化学习的C-V2X任务卸载研究

Research on C-V2X Task Offloading Based on Deep Reinforcement Learning

无人驾驶、辅助驾驶的快速发展对车辆计算性能提出了较高的要求,联合移动边缘计算的任务卸载技术可以提供解决方案。然而实现快速、高效的任务卸载决策存在巨大挑战,同时现有研究对于任务卸载的系统整体效益考虑不足。针对上述问题,采用车-路-空架构,设计一种基于软件定义网络(SDN)的蜂窝车联网(C-V2X)分布式任务卸载系统模型,并提出一种基于深度强化学习的任务卸载控制算法。对任务本地计算、边缘计算、卫星计算3种模式分别构建成本模型,以用户端车辆能耗、资源租赁费用和服务端任务处理时延、服务器负载均衡性作为联合优化目标构建目标函数。考虑任务最大期望时延、服务器最大负载率等约束,将任务卸载问题表述为混合整数非线性规划(MINLP)问题,将其建模为离散-连续混合动作空间的Markov决策过程,最后基于深度强化学习算法获得关于任务调度、资源租赁、功率控制的任务卸载决策。实验结果表明,与传统的基于粒子群优化、遗传算法的方案相比,本文算法在取得相近决策效益的同时,单次决策时延降低了45%以上。

The rapid development of driverless and assisted driving technologies has created a significant demand for enhanced vehicle computing performance.To address this demand,offloading techniques for joint Mobile Edge Computing(MEC)offer effective solutions.However,achieving fast and efficient task offloading decisions presents a significant challenge,and existing research has typically overlooked the overall system benefits associated with task offloading.To address these issues,a distributed task offloading system model for Cellular Vehicle-to-Everything(C-V2X)based on a Software-Defined Network(SDN)is designed using the vehicle-road-air architecture.A task offloading control algorithm based on Deep Reinforcement Learning(DRL)is proposed.Cost models are constructed for three modes of task:local computing,edge computing,and satellite computing.The objective function is constructed to jointly optimize two sets of criteria.On the user side,it includes vehicle energy consumption and resource leasing costs,while on the server side,it includes task processing delay and server load balance.Considering constraints such as maximum expected task delay and maximum server load ratio,the problem of task offloading is formulated as a Mixed-Integer Nonlinear Programming(MINLP)problem,which is modeled as a Markov decision process in a discrete-continuous mixed action space.Finally,task offloading decisions regarding task scheduling,resource leasing,and power control are obtained based on DRL algorithms.The experimental results show that,compared with traditional schemes based on Particle Swarm Optimization(PSO)and Genetic Algorithms(GA),the proposed algorithm achieves similar decision-making benefits while reducing the single-decision delay by more than 45%.