考虑能耗中断的无人机通信中基于深度强化学习的资源管理

Resource Management Based on Deep Reinforcement Learning for UAV Communication Considering Power-Consumption Outage

最新研究表明,高速传输导致的手机温度变化会影响相应的传输性能。针对高速传输下未考虑与手机温度有关的能耗中断而导致传输性能降低的问题,该文提出一种基于深度强化学习的资源管理方案去考虑无人机(UAV)通信场景下的能耗中断。首先,给出无人机通信的网络模型与智能手机热传递模型的分析;其次,将能耗中断的影响以约束条件的形式整合到无人机场景的优化问题中,并通过联合考虑带宽分配、功率分配和轨迹设计优化系统吞吐量;最后,采用马尔可夫决策过程描述相应的优化问题并通过名为归一化优势函数的深度强化学习算法求解。仿真表明,所提方案能有效提升系统吞吐量并得到合理的无人机飞行轨迹。

Recent research has demonstrated that the temperature variation of smartphone caused by high data rate transmission could affect the corresponding performance on transmission.Considering the problem of performance degradation on transmission caused by the ignorance of the power-consumption outage which is related with the temperature of smartphone,a deep reinforcement learning based resource management scheme is proposed to consider the power-consumption outage for Unmanned Aerial Vehicle(UAV)communication scenario.Firstly,the analysis for the network model of UAV communication and heat transfer model in smartphone is established.Then,the influence of power-consumption outage is integrated into the optimization problem of UAV scenario in the form of constraint,and the system throughput is optimized via the joint consideration of bandwidth allocation,power allocation and trajectory design.Finally,Markov decision process is adopted to depict the problem and the optimization target is achieved by a deep reinforcement learning algorithm named normalized advantage function.Simulation results manifest that the proposed scheme can effectively enhance the system throughput and achieve appropriate trajectory of UAV.