Generation and Simulation of Basic Maneuver Action Library for 6-DOF Aircraft by Reinforcement Learning

The development of modern air combat requires aircraft to have certain intelligent decision-making ability.In some of the existing solutions,the automatic control of aircraft is mostly composed of the upper mission decision and the lower control system.Although the underlying PID(Proportional Integral Derivative)based controller has a good performance in stable conditions,it lacks stability in complex environments.So,we need to design a new system for the problem of aircraft decision making.Studies have shown that the behavior of an aircraft can be viewed as a combination of several basic maneuvers.The establishment of aircraft basic motion library will effectively reduce the difficulty of upper aircraft control.Given the good performance of reinforcement learning to solve the problem with continuous action space,in this paper,reinforcement learning is used to control the aircraft’s rod and rudder to generate a basic maneuver action library,and the flight of the aircraft under the 6 degrees of freedom(6-DOF)simulation engine is effectively controlled.The simulation results verify the feasibility of the method on a visual simulation platform.

Autonomous maneuver decision-making for a UCAV in short-range aerial combat based on an MS-DDQN algorithm

To solve the problem of realizing autonomous aerial combat decision-making for unmanned combat aerial vehicles(UCAVs) rapidly and accurately in an uncertain environment, this paper proposes a decision-making method based on an improved deep reinforcement learning(DRL) algorithm: the multistep double deep Q-network(MS-DDQN) algorithm. First, a six-degree-of-freedom UCAV model based on an aircraft control system is established on a simulation platform, and the situation assessment functions of the UCAV and its target are established by considering their angles, altitudes, environments, missile attack performances, and UCAV performance. By controlling the flight path angle, roll angle, and flight velocity, 27 common basic actions are designed. On this basis, aiming to overcome the defects of traditional DRL in terms of training speed and convergence speed, the improved MS-DDQN method is introduced to incorporate the final return value into the previous steps. Finally, the pre-training learning model is used as the starting point for the second learning model to simulate the UCAV aerial combat decision-making process based on the basic training method, which helps to shorten the training time and improve the learning efficiency. The improved DRL algorithm significantly accelerates the training speed and estimates the target value more accurately during training, and it can be applied to aerial combat decision-making.

Identification and standardization of maneuvers based upon operational flight data

To find a way of loads analysis from operational flight data for advanced aircraft,maneuver identification and standardization jobs are conducted in this paper. For thousands of sorties from one aircraft, after studying the flight attitude when performing actions, the start and end time of the maneuvers can be determined. According to those time points, various types of maneuvers during the flight are extracted in the form of multi-parameters time histories. By analyzing the numerical range and curve shape of those parameters, a characteristic data library is established to model all types of maneuvers. Based on this library, a computer procedure using pattern-recognition theory is programmed to conduct automatic maneuver identification with high accuracy. In that way, operational loads are classified according to maneuver type. For a group of identified maneuvers of the same type, after the processes of time normalization, trace shifting, as well as averaging and smoothing, the idealization standard time history of each maneuver type is established.Finally, the typical load statuses are determined successfully based on standard maneuvers. The proposed method of maneuver identification and standardization is able to derive operational loads effectively, and might be applied to monitoring loads in Individual Aircraft Tracking Program（IATP）.