密集异构网络中基于强化学习的流量卸载算法

A Reinforcement Learning Algorithm for Traffic Offloading in Dense Heterogeneous Network

近年来互联网用户规模和网络流量呈现爆炸式的增长,不断逼近蜂窝移动通信网络的容量极限.流量卸载技术可充分利用现有网络,将蜂窝网络的部分流量卸载到空闲网络中,进行跨网协作实现对蜂窝网络容量的极大提升,可有效解决有限的无线带宽资源与海量高速业务需求的矛盾.将强化学习的思想引入流量卸载算法中,提出了一种异构网络中基于强化学习的流量卸载算法.该算法把流量卸载问题映射为一个强化学习问题.基于前一状态完成的动作,以WiFi网络吞吐量作为回报函数,准确地预测需卸载的流量,并计算当前网络的最大卸载量,寻找最佳的WiFi网络接入点(access point,AP),并推导出最优的流量卸载判决规则,达到异构网络整体吞吐量最大化.仿真结果表明:基于Q学习的流量卸载算法可有效地实现自适应流量卸载控制规则,有效地避免过度卸载引起的碰撞冲突和系统性能急剧恶化,达到跨网协作的负载均衡点,在保证WiFi用户服务质量的条件下,最大限度地提高LTE系统吞吐量,保证密集异构网络的整体性能.

With the explosive growth of numbers of Internet users and network traffic, the capacity of cellular mobile communication is already limited. In order to solve the contradiction between the increasing demand for high capacity and the limited resource, traffic offloading technology makes full use of the existing network, which offloads part of traffic from the cellular network into the other network and carries on the cooperation between networks, to improve the capacity of the cellular network greatly. Traffic offloading becomes one of the hot topics in the future research of wireless communication technology. In this paper, based on reinforcement learning, we propose a novel reinforcement learning algorithm for traffic offloading in dense heterogeneous network. Based on the previous experience and performance gain of the user offloading, this algorithm considers the system throughput of each state, and finds the optimal WiFi network access point （AP） by calculating the reward value. We also derive the optimal policy of traffic offloading decision to maximize the throughput of the system. Simulation results show that the reinforcement learning for traffic offloading can effectively avoid the collision caused by over offloading and rapid deterioration of system performance. Our scheme can effectively implement the adaptive traffic offloading control policy and achieve the cooperation between LTE and WiFi network guaranteeing the quality of service for users. The overall throughput of the dense heterogeneous network also reaches the maximum.