基于强化学习的无人机空战机动决策

Research on Air Combat Maneuver Decision of UAVs Based on Reinforcement Learning

针对一对一空战中无人机机动决策问题,提出了一种基于强化学习的无人机空战机动决策方法。在强化学习的框架下,分析表征空战态势的各主要因素,建立空战优势函数并以此作为强化学习回报值的基础;设计空战机动决策的动态模糊Q学习模型,对空战机动决策的状态空间进行模糊化作为强化学习的状态输入;选取典型空战动作作为强化学习基本行动,通过各模糊规则的触发强度加权求和实现连续行动空间的覆盖。相对于传统方法,本方法具有更强的鲁棒性和自主寻优性,在不断的仿真和学习中无人机所做的决策水平能够不断提高。

Aiming at the maneuver decision problem of unmanned aerial vehicle （UAV） in IV1 air combat, a maneuver decision method of UAV air combat is proposed based on reinforcement learning. Firstly, under the framework of reinforcement leaming, various factors which indicate the air combat situation are analyzed, and air combat advantage function is established as the foundation of the reward of reinforcement learning. Secondly, a dynamic fuzzy Q learning model is designed for maneuver decision in air combat, and the state space of air combat maneuver decision is then fuzzed as the state input of reinforcement learning. Finally, by selecting typical air combat maneuvers as the basic motion of reinforcement learning, the coverage of continuous action space is achieved through weighted summation of the trigger intensity of each fuzz rule. Compared with traditional methods, the method proposed is better in robustness and autonomous optimality, and by continual simulation and learning, the performance of decision made by UAV can be increasingly improved.