运营商网络中基于深度强化学习的服务功能链迁移机制

Deep Reinforcement Learning Based Migration Mechanism for Service Function Chain in Operator Networks

为改善运营商网络提供的移动服务体验,该文研究服务功能链(SFC)的在线迁移问题。首先基于马尔可夫决策过程(MDP)对服务功能链中的多个虚拟网络功能(VNF)在运营商网络中的驻留位置迁移进行模型化分析。通过将强化学习和深度神经网络相结合提出一种基于双深度Q网络(double DQN)的服务功能链迁移机制,该迁移方法能在连续时间下进行服务功能链的在线迁移决策并避免求解过程中的过度估计。实验结果表明,该文所提出的策略相比于固定部署算法和贪心算法在端到端时延和网络系统收益等方面优势明显,有助于运营商改善服务体验和资源的使用效率。

To improve the service experience provided by the operator network,this paper studies the online migration of Service Function Chain(SFC).Based on the Markov Decision Process(MDP),modeling analysis is performed on the migration of multiple Virtual Network Functions(VNF)in SFC.By combining reinforcement learning and deep neural networks,a double Deep Q-Network(double DQN)based service function chain migration mechanism is proposed.This method can make online migration decisions and avoid over-estimation.Experimental result shows that when compared with the fixed deployment algorithm and the greedy algorithm,the double DQN based SFC migration mechanism has obvious advantages in end-to-end delay and network system revenue,which can help the mobile operator to improve the quality of experience and the efficiency of resources usage.