空地网络资源分配与无人机基站动态部署算法

Resource Allocation and Dynamic Deployment Algorithm for Unmanned Aerial Vehicle Enabled Base Stations in Air-Ground Networks

为解决空地网络中地面设备数量变化引起的用户体验质量无法满足的问题,提出了一种智能网络资源分配与多无人机基站动态部署方案。首先,考虑用户体验质量和无人机、地面设备能量约束,以最小化系统总能耗为目标进行问题建模;其次,将多无人机的动态部署问题转化成具有连续动作集的马尔可夫决策过程,并根据优化目标设计了基于能耗惩罚的奖励函数;然后,采用基于确定性策略梯度的深度强化学习算法求解此问题;最后,通过仿真和对比实验验证所提方案的有效性和优越性。实验结果表明:对于海量用户场景,所提算法比深度强化学习和演员-评论家算法有更好的收敛性和更高的累积奖励,与单无人机和传统地面基站部署方案相比,所提方案系统的能耗降低约30%~40%,用户服务质量满意度提升约50%~60%。

To address the problem of unsatisfactory user experience quality caused by the fluctuation of ground device quantity in an air-ground network,a solution for intelligent network resource allocation and dynamic deployment of base stations with multiple unmanned aerial vehicles(UAVs)is proposed.Firstly,considering user experience quality and the energy constraints of UAVs and ground devices,the problem is modeled with the objective of minimizing the total system energy consumption.Secondly,the dynamic deployment of multiple UAVs is transformed into a Markov decision process(MDP)with a continuous action set,and a reward function based on energy penalty is designed according to the optimization objective.Thirdly,a deep reinforcement learning algorithm based on deep deterministic policy gradient(DDPG)is used to solve this problem.Lastly,the effectiveness and superiority of the proposed solution are verified through simulation and comparative experiment.Experimental results show that,for scenarios with a massive number of users,the proposed algorithm exhibits better convergence and higher cumulative rewards compared to deep reinforcement learning and actor-critic algorithms.In comparison to single UAV and traditional ground base station deployment solutions,the proposed solution reduces energy consumption by approximately 30%to 40%,and improves user satisfaction with service quality by around 50%to 60%.