基于Double Deep Q Network的无人机隐蔽接敌策略

A Stealthy Engagement Maneuvering Strategy of UAV Based on Double Deep Q Network

基于深度强化学习的连续状态空间无人机隐蔽接敌问题,提出了基于马尔可夫决策过程的隐蔽接敌双深度Q网络(DDQN)方法。利用DDQN生成目标值函数的方法解决了传统DQN的过拟合问题;采用按优先级随机抽样的方法获取训练样本,加速了神经网络的训练速度;设定贪婪系数按照指数下降的方法,解决了传统强化学习的“探索利用窘境”;在势函数奖赏函数设计中引入角度因子,使其更加符合实际作战情况。仿真实验结果表明,DDQN具有较好的收敛性,能有效生成隐蔽接敌策略。

This paper studies the problem of the stealthy engagement maneuvering strategy in continuous state space based on deep reinforcement learning.A stealthy engagement maneuvering strategy based on Double Deep Q Network(DDQN)by using the Markov decision process is established.The method of generating a target value function by DDQN solves the over-fitting problem of the traditional DQN.The training sample is obtained by the method of random sampling according to the priority which accelerates the training of the neural network.The greedy coefficient is set according to the method of exponential decline which solves the“exploration and utilization dilemma”of traditional reinforcement learning.An angle factor is introduced in the design of the reward function to make it more consistent with the actual combat situation.The simulation results show that DDQN has good convergence and can effectively generate the stealthy engagement maneuvering strategy.