基于深度强化学习的空中无人机基站资源分配与公平性研究

Deep reinforcement learning-based resource allocation and fairness of aerial UAV base stations

为了提高无人机基站(unmanned aerial vehicle base stations,UAV-BS)为地面多用户服务时的数据速率,提出一种基于决斗深度神经网络(dueling deep Q-network,Dueling-DQN)的深度强化学习(deep reinforcement learning,DRL)算法。采用决斗网络(dueling network,DN)结构以克服动态环境的部分可观测问题,联合优化了UAV-BS的位置和下行链路功率分配,在更符合实际的空地概率信道模型中检验了Dueling-DQN算法的性能。结果表明,相较于对比算法,所提出的Dueling-DQN算法可以提供更高的数据速率和服务公平性,且随着地面用户数量的增大,算法的优势更加明显。Dueling-DQN算法可有效解决复杂非凸性问题,为UAV-BS的资源分配问题提供理论参考。

In order to improve the data rate of unmanned aerial vehicle base stations(UAV-BS) when serving multiple users on the ground,a deep reinforcement learning(DRL) algorithm was proposed based on dueling deep Q-network(Dueling-DQN).A dueling network(DN) structure was employed to overcome the partially observable problem of the dynamic environment,and the position of the UAV-BS and the power allocation of the downlink were jointly optimized to satisfy the quality of service(QoS) of the ground users.The performance of the algorithm was examined in a more realistic air-ground probabilistic channel model.The results show that compared with the baseline algorithm,the proposed Dueling-DQN algorithm can provide higher data rate and service fairness,and the advantages are more obvious with the increase in the number of ground users.The Dueling-DQN algorithm is effective to solve the complex non-convexity problem,which provides some theoretical reference for the resource allocation problem of UAV-BS.