信息年龄受限下最小化无人机辅助无线供能网络的能耗:一种基于DQN的方法

Energy minimization in UAV⁃assisted wireless powered sensor networks with AoI constraints:A DQN⁃based approach

随着5G/B5G的不断发展,无人机在实时数据采集系统中将有广泛应用.利用无人机先给传感器节点进行无线充电,然后传感器节点利用收集到的能量将感知的信息上传无人机,可有效解决户外物联网节点的供电与数据采集问题.然而,由于无人机本身的电量受限,如何在保证无人机充电辅助物联网系统顺利完成新鲜数据采集任务的前提下最小化无人机的能耗至关重要.为此,在满足信息采集新鲜度的要求下,通过联合优化无人机的飞行时间、加速度、转角和传感器节点信息上传和能量收集调度模式,建立无人机能耗最小化优化问题.由于该问题含有整数变量,大规模情况下求解较为困难.因此,首先将其建模为马尔科夫决策过程,然后提出了一种基于DQN(Deep Q Network)的无人机能耗优化算法框架求解,并设计相对应的状态空间、动作空间和奖励函数.仿真结果验证了所提DQN算法的收敛性,同时表明提出的DQN算法比传统的贪婪算法可降低8%~30%的无人机能耗.当传感器个数超过八个时,传统的贪婪算法很难求解,而所提DQN算法仍然能找到最优解.另外,随着AoI(Age of Information)限制值的缩小或传感器数量的增加,无人机的能量消耗会不断地增加,并且由于考虑了转角约束,所提算法优化得到的无人机飞行轨迹会更平滑.

With the development of 5G/B5G,UAV(Unmanned Aerial Vehicle)will be widely employed in real⁃time data collection system.UAV can wirelessly charge ground sensors,and then sensors use the collected energy to upload the perceived information to UAV,which can effectively solve the problem of power supply and data collection in outdoor Internet of Things(IoT)systems.However,due to the limited power of UAV,how to minimize the energy consumption of UAV is very important on the premise of ensuring the freshness of collected data in UAV⁃assisted wireless powered sensor network.Therefore,this paper investigates the optimization problem of UAV's energy consumption minimization under the Age of Information(AoI)constraint by jointly optimizing UAV's flight time,acceleration,rotation angle and scheduling of information collection and energy harvesting.As the problem is a combinational optimization problem with a set of binary variables,it is difficult to be solved in large⁃scale network.Thus,it's first modeled as a Markov decision process,and a Deep Q Network(DQN)⁃based UAV's energy⁃minimal algorithm framework is proposed to solve it,and the corresponding state spaces,action spaces and reward function are designed.Simulation results demonstrate the convergence of the proposed DQN scheme,and also show that the proposed DQN scheme can reduce the UAV's energy consumption by about 8%~30%compared with the traditional greedy scheme.When the sensors'amount is more than eight,the traditional greedy scheme becomes very difficult to solve the problem,while our presented DQN method can still find an optimal solution.Moreover,with the decrease of AoI or the increment of the number of sensors,the energy consumption of UAV increases and the trajectory of UAV becomes smoother with the rotation angle constraint.