基于SumTree采样结合Double DQN的非合作式多用户动态功率控制方法

A Non-cooperative Multi-user Dynamic Power Control Method Based on SumTree Sampling and Double DQN

为了保证认知无线网络中次用户本身的通信服务质量,同时降低次用户因发射功率不合理而造成的功率损耗,提出了一种基于SumTree采样结合深度双Q网络(Double Deep Q Network,Double DQN)的非合作式多用户动态功率控制方法。通过这种方法,次用户可以不断与辅助基站进行交互,在动态变化的环境下经过不断的学习,选择以较低的发射功率完成功率控制任务。其次,该方法可以解耦目标Q值动作的选择和目标Q值的计算,能够有效减少过度估计和算法的损失。并且,在抽取经验样本时考虑到不同样本之间重要性的差异,采用了结合优先级和随机抽样的SumTree采样方法,既能保证优先级转移也能保证最低优先级的非零概率采样。仿真结果表明,该方法收敛后的算法平均损失值能稳定在0.04以内,算法的收敛速度也至少快了10个训练回合,还能提高次用户总的吞吐量上限和次用户功率控制的成功率,并且将次用户的平均功耗降低了0.5 mW以上。

To ensure the communication service quality of the secondary users in cognitive wireless networks and reduce the power loss caused by the unreasonable transmit power of the secondary users,the authors propose a non-cooperative multi-user dynamic power control method based on SumTree sampling and Double Deep Q Network(Double DQN).With this method,the secondary users can not only continuously interact with the auxiliary base station and continuously learn in a dynamically changing environment,but also choose a lower transmit power to complete the power control task.Moreover,this method can decouple the selection of the target Q-value action and the calculation of the target Q-value,which can effectively reduce overestimation and algorithm loss.In addition,it considers the importance of difference between samples when extracting empirical samples and adopts the SumTree sampling method combining priority and random sampling,which can ensure both priority transfer and non-zero probability sampling of the lowest priority.The simulation results show that the average loss value after the convergence of this method can be stabilized within 0.04.The convergence speed of the algorithm is at least 10 training rounds faster.It can also improve the total throughput upper limitation of the secondary users and the success rate of the power control in secondary users,and reduce the average power consumption for secondary users by at least 0.5 mW.