多无人机辅助数据收集系统的智能路径规划算法

Intelligent Path Planning Algorithm for Multi-UAV-assisted Data Collection Systems

无人机具有高度灵活和小巧轻便等优点,已被广泛应用于无线传感器网络的数据收集。本文考虑一个用户随机分布且处于移动状态的无线传感器网络,研究如何规划多个无人机的飞行路径以有效收集网络用户的数据。通过优化多架无人机的飞行路径,使无人机在用户位置无法预测的动态环境中实现数据收集平均吞吐量最大化,同时系统受限于无人机最短飞行时间与范围约束、无人机起点与终点约束、通信距离约束、用户通信约束和无人机防碰撞约束。使用已有优化决策方法求解该问题的计算复杂度较高,同时难以求得全局最优解。针对这一情况,本文提出一种基于Dueling Double Deep Q-network(Dueling-DDQN)的深度强化学习算法。该算法采用Dueling架构,增强算法的学习能力,提高训练过程的鲁棒性和收敛速度,同时结合了Double DQN (DDQN)算法的优势,能有效避免因过大估计Q值而导致获取次优无人机轨迹策略。仿真结果表明,此算法可以高效优化无人机的飞行路径,与已有的基准算法相比,所提算法具有更佳的收敛性和鲁棒性。

With the advantages of high flexibility and lightweight,unmanned aerial vehicles(UAVs)have been widely used in data collection of wireless sensor networks.For a multi-UAV-assisted wireless sensor network with randomly distributed and moved users,how to plan the flight paths of the UAVs to effectively collect data from the users remains a challenging problem.This paper aims to maximize the average throughput of data collection in a dynamic environment where the user's location cannot be predicted by optimizing the flight path of the UAVs,which is subject to the shortest flight time and range constraints of UAVs,the constraints of UAV start and end points,the communication distance constraints,the user communication constraints,and the UAV collision avoidance constraints.The resultant problem can be solved by using existing optimization methods with high complexity,which however is difficult to obtain the globally optimal solution.To address this problem efficiently,this paper proposes a deep reinforcement learning algorithm based on Dueling Double DQN(Dueling-DDQN).The proposed algorithm adopts the Dueling network architecture,which enhances the learning ability of the algorithm and improves the robustness and convergence speed of tracked in suboptimal solutions due to the overestimation on the Q value.Simulation results show that the proposed algorithm can efficiently obtain the flight paths of multiple UAVs under all constraints.In particular,our proposed algorithm has encouraging convergence and stability performance in comparison with the existing benchmark algorithms.