双连接场景中基于强化学习的基站选择算法

BS selection algorithm based on reinforcement learning in dual connectivity scenarios

为了满足日益增长的数据服务需求,网络运营商密集部署了大量5G小型基站。在3GPP R14中,定义了5G双连接的场景,即用户可以同时接入5G基站和4G基站。但是,目前主流的基站选择算法并不适用于这种场景。因此,为了解决5G双连接网络中的基站选择问题,文中提出一种基于强化学习的基站选择算法。该算法以用户设备为中心,以最大化用户吞吐量为目标。算法将基站选择问题映射为一个强化学习问题:将用户设备作为学习者,将无线接入技术选择策略作为动作空间,将当前时刻连入基站所获得的吞吐量作为回报值,从而计算出下一时刻选择各个基站的概率。仿真结果表明,在双连接场景中,相比于传统的RSS算法,文中算法可以减少用户设备切换次数,并提高统计时间段内用户的总吞吐量。

To meet the growing demands for data services,a large number of small 5 G base stations(BSs) have been intensively deployed by the network operators.In 3 GPP R14,a 5 G dual connectivity(DC) scenario is defined,that is,a user can simultaneously access a 5 G BS and a 4 G BS.However,the current mainstream BS selection algorithm is no longer applicable to the new scenarios.Therefore,to solve the problem of BS selection in 5 G dual-connected networks,a BS selection algorithm based on reinforcement learning is proposed.The algorithm centers on the user equipment(UE) and aims to maximize the user’s throughput.The problem of selecting BS is mapped to a reinforcement learning problem.In this problem,the user equipment is the agent,the radio access technology(RAT) selection policies are the action space,the throughput of the user at the current time is taken as a reward value.And the probability of each BS being selected at the next moment can be calculated by the reward value.The simulation results show that the proposed algorithm can reduce the number of handovers of the UE and improve the total throughput of users during the statistical time period compared with the traditional RSS algorithm.