基于深度强化学习的蜂窝无人机网络中的轨迹设计

Trajectory Design for a Cellular Internet of UAVs Using Deep Reinforcement Learning

设计了一个蜂窝无人机网络,其中无人机采集到的感知数据可以通过直通通信的方式直接传输到移动设备端,或者通过传统的蜂窝方式传输到移动设备端。由于无人机的传输模式会影响到它们的轨迹,在考虑了传输模式的情况下,研究了无人机轨迹设计问题,以最大化系统的总效用。该问题是一个状态行动空间非常大的马尔科夫决策问题,基于此问题提出了一种基于深度强化学习的多无人机轨迹设计算法。仿真结果表明所提出的算法比单智能体算法性能更好。

In this paper,we considered a cellular Internet of unmanned aerial vehicles(UAVs),in which UAVs’sensory data can be transmitted to the mobile devices directly,or through the base station by cellular communications.Since the transmission modes of UAVs may influence their trajectories,we studied the trajectory design problem for UAVs aiming to maximize the total utility in consideration of their transmission modes.Since this problem is a Markov decision problem with a large state-action space,we proposed a multi-UAV trajectory design algorithm using multi-agent deep reinforcement learning to solve this problem.Simulation results show that our proposed algorithm can achieve a higher total utility than the single-agent method.