基于Q-Learning算法的无人机空战机动决策研究

Research on UAV Air Combat Maneuver DecisionBased on Q-Learning Algorithm

针对无人机空战对抗自主机动决策问题,设计了侧向机动决策算法。通过加入启发式因子的方式和双Q表交替学习的机制,弥补了传统Q-Learning算法学习速度慢、无效学习多的不足。通过路径规划仿真和数据的对比,验证了改进Q-Learning算法具有更好的稳定性和求解能力。设计了动态的栅格规划环境,能够使无人机根据变化的空战态势自适应调整栅格尺寸大小,且对求解的速率不产生影响。基于Q-Learning算法,构建了无人机空战对抗侧向机动决策模型,并通过武器平台调换的方式验证了改进Q-Learning算法能显著提升无人机空战胜负比。

Aiming at the autonomous maneuver decision-making problem of UAV air combat,a lateral maneuver decision-making algorithm is designed.By adding heuristic factors and double Q-table alternating learning mechanism,the shortcomings of traditional Q-Learning algorithm,such as slow learning speed and many ineffective learning,are overcome.Through path planning simulation and the comparison of data,it is verified that the improved Q-Learning algorithm has better stability and solving ability.A dynamic grid planning environment is designed,which can make the UAV adjust the grid size adaptively according to the changing of air combat situation,and has no impact on the solution rate.Based on the Q-Learning algorithm,the lateral maneuver decision-making model of UAV air combat is constructed,and it is verified that the improved Q-Learning algorithm can play a significant role in improving the winning/losing ratio of UAV air combat through the exchange of weapon platforms.