基于Q学习的无人机辅助WSN数据采集轨迹规划

Trajectory Planning for Unmanned Aerial Vehicle Assisted WSN Data Collection Based on Q-Learning

针对无人机辅助采集无线传感器网络数据时各节点数据产生速率随机和汇聚节点状态不一致的场景,提出基于Q学习的非连续无人机轨迹规划算法Q-TDUD,以提高无人机能量效率和数据采集效率。基于各节点在周期内数据产生速率的随机性建立汇聚节点的汇聚延时模型,应用强化学习中的Q学习算法将各汇聚节点的延迟时间和采集链路的上行传输速率归一化到奖励函数中,通过迭代计算得到最佳非连续无人机飞行轨迹。实验结果表明,与TSP-continues、TSP、NJS-continues和NJS算法相比,Q-TDUD算法能够缩短无人机的任务完成时间,提高无人机能效和数据采集效率。

In some scenarios where UnmannedAerial Vehicle(UAV)assists inWireless Sensor Network(WSN)data collection,the data generation rate of each node is random and the states of sink node are inconsistent.To address the problem,this paper proposes a Q-learning-based algorithm called Q-TDUD for discontinuous UAV trajectory planning,which can improve the energy efficiency of UAV and data collection efficiency.Based on the randomness of the data generation rate of each node in the cycle,the aggregation delay model of the sink node is established.The Q-learning algorithm in reinforcement learning is used to normalize the delay time of each sink node and the uplink transmission rate of the collection link into the reward function,and the optimal discontinuous flight trajectory of the UAV is obtained through iterative calculation.Experimental results show that,compared with TSP-continues,TSP,NJS-continues and NJS algorithms,the proposed Q-TDUD algorithm can reduce the task completion time of UAV,and improve the energy efficiency and data collection efficiency of UAV.