基于改进型DDPG的单用户任务迁移优化

Optimization of single-user task migration based on improved DDPG

针对传统强化学习算法在具有随机任务到达和时变无线信道的边缘服务器上最小化计算成本存在收敛速度慢、收敛不稳定等问题,提出一种改进型DDPG算法(IDDPG)。将DDPG的Critic网络结构替换为Dueling结构,通过将状态价值函数拆分为优势函数和价值函数,使其收敛更快;将Critic网络的更新频率调整为高于Actor网络的更新频率,使整体训练更加稳定;在Actor网络选出来的动作上增加Ornstein-Uhlenbeck噪声以提高算法探索能力,将动作噪声大小进行分段设置,保证收敛的稳定性。实验结果表明,相较其它算法,IDDPG算法能够更好最小化计算成本,在收敛速度和收敛稳定性方面都有一定提升。

Aiming at the problems of slow convergence and unstable convergence of traditional reinforcement learning algorithms in minimizing computational cost on edge servers with random task arrivals and time-varying wireless channels,an improved DDPG algorithm(IDDPG)was proposed.The Critic network structure of DDPG was replaced by the Dueling structure,which converged faster by splitting the state value function into an advantage function and a value function.The update frequency of the Critic network was adjusted to be higher than that of the Actor network to make the overall training more stable.The Ornstein-Uhlenbeck noise was added to the actions selected through the Actor network to improve the algorithm exploration ability,and the action noise size was set in segments to ensure the stability of convergence.Experimental results show that,compared with other algorithms,the IDDPG algorithm can better minimize the computational cost,and has a certain improvement in the convergence speed and convergence stability.