基于深度强化学习的无人机路径规划与无线电测绘

UAV Path Planning and Radio Mapping Based on Deep Reinforcement Learning

针对传统无人机轨迹优化设计方法在构建通信模型上具有局限性的问题,本文面向蜂窝连接无人机通信方式,引入一种基于深度强化学习的无人机路径规划与无线电测绘方法。该方法利用扩展后的双深Q网络模型,结合无线电预测网络,生成无人机轨迹并预测由于动作选择而累计的奖励值。此外,基于Dyna框架将实际飞行和模拟飞行相结合,进一步训练双深Q网络模型,从而大大提高学习效率。仿真结果表明,与Direct-RL算法相比,该方法能更有效地利用学习到的覆盖区域概率图,使无人机避开弱覆盖区域,减小飞行时间和预期中断时间的加权和。

To address the limitations of traditional UAV trajectory optimization design methods in building communication models,this paper presents a deep reinforcement learning-based UAV path planning and radio mapping in cellular-connected UAV communication systems.The proposed method utilizes an extended double-deep Q-network(DDQN)model combined with a radio prediction network to generate UAV trajectories and predict the reward values accumulated due to action selection.Furthermore,the method trains the DDQN model by combining actual and simulated flights based on Dyna framework,which greatly improves the learning efficiency.Simulation results show that the proposed method utilizes the learned coverage area probability map more effectively compared to the Direct-RL algorithm,enabling the UAV to avoid weak coverage areas and reducing the weighted sum of flight time and expected interruption time.