基于再励学习的主动队列管理算法

A Robust Active Queue Management Algorithm Based on Reinforcement Learning

从最优决策的角度出发,将人工智能中的再励学习方法引入主动队列管理的研究中,提出了一种基于再励学习的主动队列管理算法RLGD(reinforcementlearninggradient-descent).RLGD以速率匹配和队列稳定为优化目标,根据网络状态自适应地调节更新步长,使得队列长度能够很快收敛到目标值,并且抖动很小.此外,RLGD不需要知道源端的速率调整算法,因而具有很好的可扩展性.通过不同网络环境下的仿真显示,RLGD与REM,PI等AQM算法相比,具有更好的性能和鲁棒性.

From the viewpoint of decision theory, AQM (active queue management) can be considered as an optimal decision problem. In this paper, a new AQM scheme, Reinforcement Learning Gradient-Descent (RLGD), is described based on the optimal decision theory of reinforcement learning. Aiming to maximize the throughput and stabilize the queue length, RLGD adjusts the update step adaptively, without the demand of knowing the rate adjustment scheme of the source sender. Simulation demonstrates that RLGD can lead to the convergence of the queue length to the desired value quickly and maintain the oscillation small. The results also show that the RLGD scheme is very robust to disturbance under various network conditions and outperforms the traditional REM and PI controllers significantly.