基于深度强化学习的转发效能感知流量调度算法

Forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning

软件定义网络(SDN)通过将控制平面与数据平面分离,可实现灵活的流量调度,更有效地利用网络资源。但是,流表项数量、设备负载率及连接主机数量增加等因素的共同作用会导致SDN交换机的转发效能降低,进而影响端到端的数据传输时延。为了解决上述问题,提出了基于深度强化学习的转发效能感知流量调度算法。首先,将交换机状态信息统一到感知模型中,通过神经网络建立交换机状态信息和转发效能的映射关系。然后,结合网络状态和流量信息,通过深度强化学习产生流量调度策略。最后,通过由最短路径和负载均衡算法产生的专家样本引导模型训练,不仅使模型学习到专家样本的知识以提升性能,同时提升模型训练效率。实验结果表明,与其他算法相比,所提算法不仅使端到端的平均传输时延降低了15.31%,而且保证了网络整体的负载均衡。

The software defined network separates the control plane from the data plane to achieve flexible traffic scheduling,which can use network resources more efficiently.However,with the increase of the number of flow entries,load rate,the number of connected hosts,and other factors,the forwarding efficiency of the SDN switch will be reduced,which will affect the end-to-end transmission delay.To solve the above problems,the forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning was proposed.First,the switch state was integrated into the perception model,and the mapping relationship between switch state information and forwarding efficiency was established based on neural network.Then,combined with network state and traffic information,traffic scheduling policy was generated through deep reinforce-ment learning.Finally,the expert samples generated by the shortest path and load balance algorithms could guide the model training,which enabled the model to learn knowledge from expert samples to improve performance and accelerated the training process.The experimental results show that the proposed algorithm not only reduces the average end-to-end trans-mission delay by 15.31%,but also ensures the overall load balance of the network,compared with other algorithms.