基于预训练深度强化学习的星地网络SDN部署策略

SDN deployment strategy for satellite⁃ground network based on pre⁃trained deep reinforcement learning

万物互联和天地一体化网络的趋势下,全球覆盖的卫星接入地面骨干网络将成为未来的热点。传统的由先验知识去部署SDN节点的方法已不再适用高速变动的星地网络。经过研究,给出一种基于预训练的深度确定性策略梯度(P⁃DDPG)的SDN节点部署方法,把高速变化的拓扑结构分成相对静止的拓扑结构。在传统深度强化学习的基础上,通过迁移预训练中的权重及经验缓冲池来提高Actor⁃Critic网络的学习效率,得出SDN节点部署策略。实验结果表明,P⁃DDPG算法可以有效地在星地网络拓扑变化之后给出SDN节点部署策略,提高网络整体的负载均衡水平。

Under the trend of the Internet of Everything and the integrated network of space and earth,the global coverage of satellite access to the ground backbone network will become a hot spot in the future.The traditional method of deploying SDN nodes based on prior knowledge is no longer suitable for high⁃speed changing satellite⁃to⁃ground networks.A pre⁃trained deep de⁃terministic policy gradient(P⁃DDPG)based SDN node deployment method is proposed in this work and the high⁃speed changing to⁃pology was into divided into relatively static topologies.On the basis of traditional deep reinforcement learning,the learning effi⁃ciency of the Actor⁃Critic network is improved by migrating the weights and experience buffer pools in the pre⁃training,and the SDN node deployment strategy is obtained.The experimental results show that the P⁃DDPG algorithm proposed in this paper can ef⁃fectively provide the SDN node deployment strategy after the satellite⁃ground network topology changes,and improve the overall load balancing level of the network.