基于改进Dueling DQN的无人机网络多域资源分配策略

A Multidomain Resource Allocation Strategy for UAV Network Based on Improved Dueling DQN

针对无人机(Unmanned Aerial Vehicle,UAV)网络节点间干扰导致通信质量降低问题,文中从空域与功率域出发,构建以最大化UAV系统频谱效率和能量效率为目标的波束成形与发射功率联合优化模型。针对该多域资源分配模型,提出参数化动作空间的加权Dueling DQN(P-wDDQN)学习算法,该算法适用于包含连续功率分配和离散波束成形的混合动作空间,且解决了Dueling DQN的目标Q值过低估计问题。利用所提P-wDDQN算法设计了联合波束与功率资源分配策略。仿真结果表明,所提多域资源分配策略提升了无人机网络的频谱效率和能量效率,且具有快速收敛的优势。

To address the problem of communication quality degradation due to inter-link interference in unmanned aerial vehicle(UAV)network,this paper constructs a joint beamforming and transmit power optimization model with the objective of maximizing the spectral efficiency and energy efficiency of UAV systems from the space and power domains.A weighted Dueling DQN learning algorithm based on parameterized action space(P-wDDQN)is proposed for the multidomain resource allocation model.This algorithm is suitable for mixed action spaces containing continuous power allocation and discrete beamforming,and solves the problem of overestimating the target Q value of Dueling DQN.A beamforming and power resource allocation strategy was designed using the proposed P-wDDQN algorithm.Simulation results show that the proposed multidomain resource allocation strategy improves the spectral efficiency and energy efficiency of UAV network and has the advantage of rapid convergence.