基于深度强化学习的无人机航路规划算法研究

Research on UAV Path Planning Method Based on Deep Reinforcement Learning

航路规划是无人机在复杂战场环境中完成作战任务的关键技术之一。本文提出了一种基于PER-D3QN的无人机航路规划算法,通过网络模型设计、状态空间设计、动作空间设计和收益函数设计实现无人机在战场环境下的航路规划。PER-D3QN算法将目标网络模型、竞争网络模型和优先级经验重现策略进行结合,有效地解决了深度强化学习方法存在的过拟合问题和网络优化不稳定问题。最后,通过仿真试验验证了所提算法相较于Double DQN和DQN算法具有更好的收敛性、稳定性和适用性,相较于A*算法具有较好的实时性,可高效实现无人机在复杂战场环境下的航路规划,有效帮助无人机遂行作战任务。

Path planning is one of the key technologies for UAVs to accomplish operation missions in complex battlefield environments.In this paper,we propose a UAV path planning method based on PER-D3QN,which realizes the path planning for UAVs in the battlefield environment through network model design,state space design,action space design and reward function design.The PER-D3QN algorithm combines the target network,dueling network and prioritized experience replay,which effectively solves the overfitting problem and unstable problem in deep reinforcement learning.In the end,it is verified through simulation experiments that the proposed method achieved better convergence,stability and applicability compared with double DQN and DQN algorithms,and better real-time performance compared with A*algorithm,which can efficiently realize the path planning of UAVs in the complex battlefield environment,and effectively help the UAVs to attempt the operational mission.