UAV cooperative air combat maneuver decision based on multi-agent reinforcement learning

In order to improve the autonomous ability of unmanned aerial vehicles(UAV)to implement air combat mission,many artificial intelligence-based autonomous air combat maneuver decision-making studies have been carried out,but these studies are often aimed at individual decision-making in 1 v1 scenarios which rarely happen in actual air combat.Based on the research of the 1 v1 autonomous air combat maneuver decision,this paper builds a multi-UAV cooperative air combat maneuver decision model based on multi-agent reinforcement learning.Firstly,a bidirectional recurrent neural network(BRNN)is used to achieve communication between UAV individuals,and the multi-UAV cooperative air combat maneuver decision model under the actor-critic architecture is established.Secondly,through combining with target allocation and air combat situation assessment,the tactical goal of the formation is merged with the reinforcement learning goal of every UAV,and a cooperative tactical maneuver policy is generated.The simulation results prove that the multi-UAV cooperative air combat maneuver decision model established in this paper can obtain the cooperative maneuver policy through reinforcement learning,the cooperative maneuver policy can guide UAVs to obtain the overall situational advantage and defeat the opponents under tactical cooperation.

Extracting Decision Rules for Cooperative Team Air Combat Based on Rough Set Theory

In order to reduce redundant features in air combat information and to meet the requirements of real-time decision in combat, rough set theory is introduced to the tactical decision analysis in cooperative team air combat. An algorithm of attribute reduction for extracting key combat information and generating tactical rules from given air combat databases is presented. Then, considering the practical requirements of team combat, a method for reduction of attribute-values under single decision attribute is extended to the reduction under multi-decision attributes. Finally, the algorithm is verified with an example for tactical choices in team air combat. The results show that, the redundant attributes in air combat information can be reduced, and that the main combat attributes, i.e., the information about radar command and medium-range guided missile, can be obtained with the algorithm mentioned above, moreover, the minimal reduced strategy for tactical decision can be generated without losing the result of key information classification. The decision rules extracted agree with the real situation of team air combat.

Target distribution in cooperative combat based on Bayesian optimization algorithm

Target distribution in cooperative combat is a difficult and emphases. We build up the optimization model according to the rule of fire distribution. We have researched on the optimization model with BOA. The BOA can estimate the joint probability distribution of the variables with Bayesian network, and the new candidate solutions also can be generated by the joint distribution. The simulation example verified that the method could be used to solve the complex question, the operation was quickly and the solution was best.

Task assignment under constraint of timing sequential for cooperative air combat

According to the previous achievement, the task assignment under the constraint of timing continuity for a cooperative air combat is studied. An extensive task assignment scenario with the background of the cooperative air combat is proposed. The utility and time of executing a task as well as the continuous combat ability are defined. The concept of the matching method of weapon and target is modified based on the analysis of the air combat scenario. The constraint framework is also redefined according to a new objective function. The constraints of timing and continuity are formulated with a new method, at the same time, the task assignment and integer programming models of the cooperative combat are established. Finally, the assignment problem is solved using the integrated linear programming software and the simulation shows that it is feasible to apply this modified model in the cooperative air combat for tasks cooperation and it is also efficient to optimize the resource assignment.

Autonomous maneuver strategy of swarm air combat based on DDPG

Unmanned aerial vehicles(UAVs)have been found significantly important in the air combats,where intelligent and swarms of UAVs will be able to tackle with the tasks of high complexity and dynamics.The key to empower the UAVs with such capability is the autonomous maneuver decision making.In this paper,an autonomous maneuver strategy of UAV swarms in beyond visual range air combat based on reinforcement learning is proposed.First,based on the process of air combat and the constraints of the swarm,the motion model of UAV and the multi-to-one air combat model are established.Second,a two-stage maneuver strategy based on air combat principles is designed which include inter-vehicle collaboration and target-vehicle confrontation.Then,a swarm air combat algorithm based on deep deterministic policy gradient strategy(DDPG)is proposed for online strategy training.Finally,the effectiveness of the proposed algorithm is validated by multi-scene simulations.The results show that the algorithm is suitable for UAV swarms of different scales.

Cooperative Threat Assessment of Multi-aircrafts Based on Synthetic Fuzzy Cognitive Map

Threat assessment is one of the most important parts of the tactical decisions,and it has a very important influence on task allocation.An application of fuzzy cognitive map(FCM) for target threat assessment in the air combat is introduced.Considering the fact that the aircrafts participated in the cooperation may not have the same threat assessment mechanism,two different FCM models are established.Using the method of combination,the model of cooperative threat assessment in air combat of multi-aircrafts is established.Simulation results show preliminarily that the method is reasonable and effective.Using FCM for threat assessment is feasible.