一种车载服务的快速深度Q学习网络边云迁移策略

A Fast Deep Q-learning Network Edge Cloud Migration Strategy for Vehicular Service

智能网联交通系统中车载用户的高速移动,不可避免地造成了数据在边缘服务器之间频繁迁移,产生了额外的通信回传时延,对边缘服务器的实时计算服务带来了巨大的挑战。为此,该文提出一种基于车辆运动轨迹的快速深度Q学习网络(DQN-TP)边云迁移策略,实现数据迁移的离线评估和在线决策。车载决策神经网络实时获取接入的边缘服务器网络状态和通信回传时延,根据车辆的运动轨迹进行虚拟机或任务迁移的决策,同时将实时的决策信息和获取的边缘服务器网络状态信息发送到云端的经验回放池中;评估神经网络在云端读取经验回放池中的相关信息进行网络参数的优化训练,定时更新车载决策神经网络的权值,实现在线决策的优化。最后仿真验证了所提算法与虚拟机迁移算法和任务迁移算法相比能有效地降低时延。

The high-speed movement of vehicles inevitably leads to frequent data migration between edge servers and increases communication delay,which brings great challenges to the real-time computing service of edge servers.To solve this problem,a real-time reinforcement learning method based on Deep Q-learning Networks according to vehicle motion Trajectory Process(DQN-TP)is proposed.The proposed algorithm separates the decision-making process from the training process by using two neural networks.The decision neural network obtains the network state in real time according to the vehicle’s movement track and chooses the migration method in the virtual machine migration and task migration.At the same time,the decision neural network uploads the decision records to the memory replay pool in the cloud.The evaluation neural network in the cloud trains with the records in the memory replay pool and periodically updates the parameters to the on-board decision neural network.In this way,training and decision-making can be carried out simultaneously.At last,a large number of simulation experiments show that the proposed algorithm can effectively reduce the latency compared with the existing methods of task migration and virtual machine migration.