基于深度强化学习的无人机切换管理研究

Research on UAV Handover Management Based on Deep Reinforcement Learning

为无人机提供网络连接是未来蜂窝网络系统的一个主要应用,无人机在蜂窝网络中作为移动基站或移动用户设备时,需要在不同的基站之间切换,以保持高速可靠的网络连接。针对无人机移动性强、飞行环境复杂造成无人机在蜂窝基站间发生频繁切换、切换失败等问题,提出了一种基于深度强化学习的无人机连接蜂窝网络切换优化方法。基于深度强化学习框架,实现无人机自适应基站切换的在线学习和决策,克服了以往算法中当状态空间过大而导致训练时间长、泛化能力差的缺点;融合参考信号接收功率和切换次数两项指标作为联合奖励函数,保证无人机在稳定蜂窝网络连接的前提下,减少了无人机在蜂窝基站间的无效切换次数。实验结果表明,所提出的算法经过1000轮训练,无人机的平均切换次数显著降低,有效避免了不必要的切换,降低了切换失败的概率,提升了无人机连接蜂窝网络时的信号接收功率。

Providing network connections for drones is a major application of future cellular network systems.When drones serve as mobile base stations or mobile user equipment in cellular networks,they need to switch between different base stations to maintain high-speed and reliable network connections.Aiming at the problems of frequent handovers and handover failures of UAVs between cellular base stations caused by high mobility of UAVs and complex flight environment,a method for optimizing handover of UAVs connected to cellular networks based on deep reinforcement learning is proposed.First of all,based on a deep reinforcement learning framework,online learning and decision-making for adaptive base station switching of UAVs are realized,which overcomes the shortcomings of previous algorithms that result in long training time and poor generalization ability when the state space is too large.Secondly,two indicators of reference signal received power and handover times are integrated as a joint reward function to ensure that the UAV has a stable cellular network connection and reduces the number of invalid handovers between the UAV and the cellular base station.Experimental results show that after 1000 rounds of training,the proposed algorithm has significantly reduced the average number of handovers for UAV,effectively avoiding unnecessary handovers,reducing the probability of handover failures,and improving the receive power of UAV when connecting to cellular networks.