基于改进MDP的边缘计算任务卸载研究

Research on Edge Computing Task Offloading Based on Improved Markov Decision Process

针对强化学习进行边缘计算任务卸载时,面临大规模动作空间导致的收敛慢,计算速率低的问题,采用改进马尔科夫决策过程(Markov Decision Process, MDP)的移动边缘计算任务卸载算法。首先对信道增益去噪,使用时间卷积网络(Temporal convolutional network, TCN)生成卸载动作,然后根据改进的MDP选择最优卸载动作,引入经验回放机制存储最优卸载动作,依据提出的概率优先级抽样训练TCN,最终得到最佳卸载策略与资源分配。实验对比任务全部本地处理、全部卸载、长短期记忆网络融合改进MDP等基准算法,由结果得出模型可以快速收敛,有效提高计算速率,证明了模型的有效性和可靠性。

Aiming at the problems of slow convergence and low computing speed caused by large-scale action space when reinforcement learning is used for edge computing task offloading,a mobile edge computing task offloading algorithm based on Improved Markov decision process(MDP)was adopted.Firstly,the channel gain was denoised,and the temporal convolutional network(TCN)was used to generate the offloading action.Then,the optimal offloading action was selected according to the improved MDP,the empirical playback mechanism was introduced to store the optimal offloading action,and the TCN was trained according to the proposed probability priority sampling.Finally,the optimal offloading strategy and resource allocation were obtained.The experimental results show that the model can converge quickly and effectively improve the calculation rate,which proves the effectiveness and reliability of the model.