基于PPO算法的无人机近距空战自主引导方法

An Autonomous Guidance Method of UAV in Close Air Combat Based on PPO Algorithm

针对无人机近距空战的自主决策问题,提出了一种基于近端策略优化(PPO)算法的无人机自主引导方法。针对敌我距离、角度、速度以及任务约束等信息重塑奖励,建立了无人机三自由度模型,在速度坐标系上构建强化学习的状态和动作,分别对结合了全连接神经网络的PPO算法(标准PPO算法)和长短时记忆网络的PPO算法(改进PPO算法)模型进行了仿真训练。根据训练的结果可以证明,相比于标准PPO算法,所提的改进PPO算法能够更有效地处理与时间序列高度相关的无人机自主引导任务。

Aiming at the problem of UAV’s autonomous decision-making in close air combat,an autonomous guidance method for UAV based on Proximal Policy Optimization(PPO) algorithm is proposed.The rewards are reshaped,such as distance,angle,speed and mission constraint,a three-degree-of-freedom model of UAV is established,and the state and action of reinforcement learning are constructed on the velocity coordinate system.The simulation training is carried out on the model of PPO algorithm combined with the fully connected neural network(standard PPO algorithm) and the PPO algorithm combined with the long short-term memory network(improved PPO algorithm) respecitively.According to the training results,it can be proved that,compared with the standard PPO algorithm,the improved PPO algorithm proposed in this paper can handle the UAV autonomous guidance tasks that are highly correlated with time series more effectively.