Weapon configuration, allocation and route planning with time windows for multiple unmanned combat air vehicles

Unmanned combat air vehicles(UCAVs) mission planning is a fairly complicated global optimum problem. Military attack missions often employ a fleet of UCAVs equipped with weapons to attack a set of known targets. A UCAV can carry different weapons to accomplish different combat missions. Choice of different weapons will have different effects on the final combat effectiveness. This work presents a mixed integer programming model for simultaneous weapon configuration and route planning of UCAVs, which solves the problem optimally using the IBM ILOG CPLEX optimizer for simple missions. This paper develops a heuristic algorithm to handle the medium-scale and large-scale problems. The experiments demonstrate the performance of the heuristic algorithm in solving the medium scale and large scale problems. Moreover, we give suggestions on how to select the most appropriate algorithm to solve different scale problems.

Formation and adjustment of manned/unmanned combat aerial vehicle cooperative engagement system

Manned combat aerial vehicles （MCAVs）, and un-manned combat aerial vehicles （UCAVs） together form a cooper-ative engagement system to carry out operational mission, whichwill be a new air engagement style in the near future. On the basisof analyzing the structure of the MCAV/UCAV cooperative engage-ment system, this paper divides the unique system into three hi-erarchical levels, respectively, i.e., mission level, task-cluster leveland task level. To solve the formation and adjustment problem ofthe latter two levels, three corresponding mathematical modelsare established. To solve these models, three algorithms calledquantum artificial bee colony （QABC） algorithm, greedy strategy（GS） and two-stage greedy strategy （TSGS） are proposed. Finally,a series of simulation experiments are designed to verify the effec-tiveness and superiority of the proposed algorithms.

A Multi-UCAV cooperative occupation method based on weapon engagement zones for beyond-visual-range air combat

Recent advances in on-board radar and missile capabilities,combined with individual payload limitations,have led to increased interest in the use of unmanned combat aerial vehicles(UCAVs)for cooperative occupation during beyond-visual-range(BVR)air combat.However,prior research on occupational decision-making in BVR air combat has mostly been limited to one-on-one scenarios.As such,this study presents a practical cooperative occupation decision-making methodology for use with multiple UCAVs.The weapon engagement zone(WEZ)and combat geometry were first used to develop an advantage function for situational assessment of one-on-one engagement.An encircling advantage function was then designed to represent the cooperation of UCAVs,thereby establishing a cooperative occupation model.The corresponding objective function was derived from the one-on-one engagement advantage function and the encircling advantage function.The resulting model exhibited similarities to a mixed-integer nonlinear programming(MINLP)problem.As such,an improved discrete particle swarm optimization(DPSO)algorithm was used to identify a solution.The occupation process was then converted into a formation switching task as part of the cooperative occupation model.A series of simulations were conducted to verify occupational solutions in varying situations,including two-on-two engagement.Simulated results showed these solutions varied with initial conditions and weighting coefficients.This occupation process,based on formation switching,effectively demonstrates the viability of the proposed technique.These cooperative occupation results could provide a theoretical framework for subsequent research in cooperative BVR air combat.

Autonomous maneuver decision-making for a UCAV in short-range aerial combat based on an MS-DDQN algorithm

To solve the problem of realizing autonomous aerial combat decision-making for unmanned combat aerial vehicles(UCAVs) rapidly and accurately in an uncertain environment, this paper proposes a decision-making method based on an improved deep reinforcement learning(DRL) algorithm: the multistep double deep Q-network(MS-DDQN) algorithm. First, a six-degree-of-freedom UCAV model based on an aircraft control system is established on a simulation platform, and the situation assessment functions of the UCAV and its target are established by considering their angles, altitudes, environments, missile attack performances, and UCAV performance. By controlling the flight path angle, roll angle, and flight velocity, 27 common basic actions are designed. On this basis, aiming to overcome the defects of traditional DRL in terms of training speed and convergence speed, the improved MS-DDQN method is introduced to incorporate the final return value into the previous steps. Finally, the pre-training learning model is used as the starting point for the second learning model to simulate the UCAV aerial combat decision-making process based on the basic training method, which helps to shorten the training time and improve the learning efficiency. The improved DRL algorithm significantly accelerates the training speed and estimates the target value more accurately during training, and it can be applied to aerial combat decision-making.

Trajectory online optimization for unmanned combat aerial vehicle using combined strategy

This paper presents a combined strategy to solve the trajectory online optimization problem for unmanned combat aerial vehicle (UCAV). Firstly, as trajectory directly optimizing is quite time costing, an online trajectory functional representation method is proposed. Considering the practical requirement of online trajectory, the 4-order polynomial function is used to represent the trajectory, and which can be determined by two independent parameters with the trajectory terminal conditions; thus, the trajectory online optimization problem is converted into the optimization of the two parameters, which largely lowers the complexity of the optimization problem. Furthermore, the scopes of the two parameters have been assessed into small ranges using the golden section ratio method. Secondly, a multi-population rotation strategy differential evolution approach (MPRDE) is designed to optimize the two parameters; in which, 'current-to-best/1/bin', 'current-to-rand/1/bin' and 'rand/2/bin' strategies with fixed parameter settings are designed, these strategies are rotationally used by three subpopulations. Thirdly, the rolling optimization method is applied to model the online trajectory optimization process. Finally, simulation results demonstrate the efficiency and real-time calculation capability of the designed combined strategy for UCAV trajectory online optimizing under dynamic and complicated environments.

UCAV situation assessment method based on C-LSHADE-Means and SAE-LVQ

The unmanned combat aerial vehicle(UCAV)is a research hot issue in the world,and the situation assessment is an important part of it.To overcome shortcomings of the existing situation assessment methods,such as low accuracy and strong dependence on prior knowledge,a datadriven situation assessment method is proposed.The clustering and classification are combined,the former is used to mine situational knowledge,and the latter is used to realize rapid assessment.Angle evaluation factor and distance evaluation factor are proposed to transform multi-dimensional air combat information into two-dimensional features.A convolution success-history based adaptive differential evolution with linear population size reduc-tion-means(C-LSHADE-Means)algorithm is proposed.The convolutional pooling layer is used to compress the size of data and preserve the distribution characteristics.The LSHADE algorithm is used to initialize the center of the mean clustering,which over-comes the defect of initialization sensitivity.Comparing experi-ment with the seven clustering algorithms is done on the UCI data set,through four clustering indexes,and it proves that the method proposed in this paper has better clustering performance.A situation assessment model based on stacked autoen-coder and learning vector quantization(SAE-LVQ)network is constructed,and it uses SAE to reconstruct air combat data fea-tures,and uses the self-competition layer of the LVQ to achieve efficient classification.Compared with the five kinds of assess-ments models,the SAE-LVQ model has the highest accuracy.Finally,three kinds of confrontation processes from air combat maneuvering instrumentation(ACMI)are selected,and the model in this paper is used for situation assessment.The assessment results are in line with the actual situation.

Cooperative decision-making algorithm with efficient convergence for UCAV formation in beyond-visual-range air combat based on multi-agent reinforcement learning

Highly intelligent Unmanned Combat Aerial Vehicle(UCAV)formation is expected to bring out strengths in Beyond-Visual-Range(BVR)air combat.Although Multi-Agent Reinforcement Learning(MARL)shows outstanding performance in cooperative decision-making,it is challenging for existing MARL algorithms to quickly converge to an optimal strategy for UCAV formation in BVR air combat where confrontation is complicated and reward is extremely sparse and delayed.Aiming to solve this problem,this paper proposes an Advantage Highlight Multi-Agent Proximal Policy Optimization(AHMAPPO)algorithm.First,at every step,the AHMAPPO records the degree to which the best formation exceeds the average of formations in parallel environments and carries out additional advantage sampling according to it.Then,the sampling result is introduced into the updating process of the actor network to improve its optimization efficiency.Finally,the simulation results reveal that compared with some state-of-the-art MARL algorithms,the AHMAPPO can obtain a more excellent strategy utilizing fewer sample episodes in the UCAV formation BVR air combat simulation environment built in this paper,which can reflect the critical features of BVR air combat.The AHMAPPO can significantly increase the convergence efficiency of the strategy for UCAV formation in BVR air combat,with a maximum increase of 81.5%relative to other algorithms.

Optimal Formation Reconfiguration Control of Multiple UCAVs Using Improved Particle Swarm Optimization

Optimal formation reconfiguration control of multiple Uninhabited Combat Air Vehicles （UCAVs） is a complicated global optimum problem. Particle Swarm Optimization （PSO） is a population based stochastic optimization technique inspired by social behaviour of bird flocking or fish schooling. PSO can achieve better results in a faster, cheaper way compared with other bio-inspired computational methods, and there are few parameters to adjust in PSO. In this paper, we propose an improved PSO model for solving the optimal formation reconfiguration control problem for multiple UCAVs. Firstly, the Control Parameterization and Time Diseretization （CPTD） method is designed in detail. Then, the mutation strategy and a special mutation-escape operator are adopted in the improved PSO model to make particles explore the search space more efficiently. The proposed strategy can produce a large speed value dynamically according to the variation of the speed, which makes the algorithm explore the local and global minima thoroughly at the same time. Series experimental results demonstrate the feasibility and effectiveness of the proposed method in solving the optimal formation reconfiguration control problem for multiple UCAVs.

Deep reinforcement learning and its application in autonomous fitting optimization for attack areas of UCAVs

The ever-changing battlefield environment requires the use of robust and adaptive technologies integrated into a reliable platform. Unmanned combat aerial vehicles(UCAVs) aim to integrate such advanced technologies while increasing the tactical capabilities of combat aircraft. As a research object, common UCAV uses the neural network fitting strategy to obtain values of attack areas. However, this simple strategy cannot cope with complex environmental changes and autonomously optimize decision-making problems. To solve the problem, this paper proposes a new deep deterministic policy gradient(DDPG) strategy based on deep reinforcement learning for the attack area fitting of UCAVs in the future battlefield. Simulation results show that the autonomy and environmental adaptability of UCAVs in the future battlefield will be improved based on the new DDPG algorithm and the training process converges quickly. We can obtain the optimal values of attack areas in real time during the whole flight with the well-trained deep network.

Autonomous Maneuver Decisions via Transfer Learning Pigeon-Inspired Optimization for UCAVs in Dogfight Engagements

This paper proposes an autonomous maneuver decision method using transfer learning pigeon-inspired optimization(TLPIO)for unmanned combat aerial vehicles(UCAVs)in dogfight engagements.Firstly,a nonlinear F-16 aircraft model and automatic control system are constructed by a MATLAB/Simulink platform.Secondly,a 3-degrees-of-freedom(3-DOF)aircraft model is used as a maneuvering command generator,and the expanded elemental maneuver library is designed,so that the aircraft state reachable set can be obtained.Then,the game matrix is composed with the air combat situation evaluation function calculated according to the angle and range threats.Finally,a key point is that the objective function to be optimized is designed using the game mixed strategy,and the optimal mixed strategy is obtained by TLPIO.Significantly,the proposed TLPIO does not initialize the population randomly,but adopts the transfer learning method based on Kullback-Leibler(KL)divergence to initialize the population,which improves the search accuracy of the optimization algorithm.Besides,the convergence and time complexity of TLPIO are discussed.Comparison analysis with other classical optimization algorithms highlights the advantage of TLPIO.In the simulation of air combat,three initial scenarios are set,namely,opposite,offensive and defensive conditions.The effectiveness performance of the proposed autonomous maneuver decision method is verified by simulation results.

Threat sequencing of multiple UCAVs with incomplete information based on game theory

The threat sequencing of multiple unmanned combat air vehicles(UCAVs) is a multi-attribute decision-making(MADM)problem. In the threat sequencing process of multiple UCAVs,due to the strong confrontation and high dynamics of the air combat environment, the weight coefficients of the threat indicators are usually time-varying. Moreover, the air combat data is difficult to be obtained accurately. In this study, a threat sequencing method of multiple UCAVs is proposed based on game theory by considering the incomplete information. Firstly, a zero-sum game model of decision maker( D) and nature(N)with fuzzy payoffs is established to obtain the uncertain parameters which are the weight coefficient parameters of the threat indicators and the interval parameters of the threat matrix. Then,the established zero-sum game with fuzzy payoffs is transformed into a zero-sum game with crisp payoffs(matrix game) to solve. Moreover, a decision rule is addressed for the threat sequencing problem of multiple UCAVs based on the obtained uncertain parameters. Finally, numerical simulation results are presented to show the effectiveness of the proposed approach.

Integrated guidance and control design of the suicide UCAV for terminal attack

A novel integrated guidance and control (IGC) design method is proposed to solve problems of low control accuracy for a suicide unmanned combat aerial vehicle (UCAV) in the terminal attack stage. First of all, the IGC system model of the UCAV is built based on the three-channel independent design idea, which reduces the difficulties of designing the controller. Then, IGC control laws are designed using the trajectory linearization control (TLC). A nonlinear disturbance observer (NDO) is introduced to the IGC controller to reject various uncertainties, such as the aerodynamic parameter perturbation and the measurement error interference. The stability of the closed-loop system is proven by using the Lyapunov theorem. The performance of the proposed IGC design method is verified in a terminal attack mission of the suicide UCAV. Finally, simulation results demonstrate the superiority and effectiveness in the aspects of guidance accuracy and system robustness.

Cooperative Path Dynamic Planning Model of UCAV Team Based on Global Optimization Method

Cooperative path dynamic planning of a UCAV （unmanned combat air vehicle） team not only considers the capability of task requirement of single UCAV, but also considers the cooperative dynamic connection among members of the UCAV team. A cooperative path dynamic planning model of the UCAV team by applying a global optimization method is discussed in this paper and the corresponding model is built and analyzed. By the example simulation, the reasonable result acquired indicates that the model could meet dynamic planning demand under the circumstance of membership functions. The model is easy to be realized and has good practicability.

Computational engineering analysis of external geometrical modifications on MQ-1 unmanned combat aerial vehicle

This paper focuses on the effects of external geometrical modifications on the aerodynamic characteristics of the MQ-1 predator Unmanned Combat Aerial Vehicle(UCAV)using computational fluid dynamics.The investigations are performed for 16 flight conditions at an altitude of7.6 km and at a constant speed of 56.32 m/s.Two models are analysed,namely the baseline model and the model with external geometrical modifications installed on it.Both the models are investigated for various angles of attack from-4°to 16°,angles of bank from 0°to 6°and angles of yaw from 0°to 4°.Due to the unavailability of any experimental(wind tunnel or flight test)data for this UCAV in the literature,a thorough verification of calculations process is presented to demonstrate confidence level in the numerical simulations.The analysis quantifies the loss of lift and increase in drag for the modified version of the MQ-1 predator UCAV along with the identification of stall conditions.Local improvement(in drag)of up to 96%has been obtained by relocating external modifications,whereas global drag force reduction of roughly 0.5%is observed.The effects of external geometrical modifications on the control surfaces indicate the blanking phenomenon and reduction in forces on the control surfaces that can reduce the aerodynamic performance of the UCAV.

Monte Carlo Based Analysis and Validation Method for Time for Single Vehicle Combat Preparation of Armored Vehicle

Time for single vehicle combat preparation(TSVCP) is an important characteristic parameter for the operational support feature of armored vehicle. During the development phase, how to validate the TSVCP of armored vehicle through analytic method is a difficult issue in analysis and validation of vehicle supportability.This paper uses Monte Carlo approach and builds a working model for single vehicle combat preparation(SVCP)of armored vehicle, thus realizes the prediction and analysis of the TSVCP of armored vehicle, and finally validates the effectiveness of the approach by example.

Key Parameters and Conceptual Configuration of Unmanned Combat Aerial Vehicle Concept

The nature and characteristics of attack unmanned combat aerial vehicle （UCAV） are analyzed. The principles of selecting takeoff thrust-weight ratio and takeoff weight of attack UCAV are presented by analyzing the statistical data of weights for various main combat aircraft. The UCAV airborne weapons are analyzed, followed by the preliminary estimation of the payload weight. Various typical engines are analyzed and one of them is selected. Then the takeoff weight of the UCAV is determined. Based on some basic parameters and assumptions, the qualitative decomposition calculation for takeoff weight is completed. The key factors for obtaining longer endurance of aircraft with small aspect ratio configuration are found to be high lift-drag ratio and internal space. On the basis of the conclusions mentioned above, a highly blended flying-wing plus lifting body concept is proposed. According to this concept, the UCAV configuration is designed and optimized. Finally, the UCAV configuration with small aspect ratio, high lift-drag ratio, and high stealth characteristic is obtained.

Real-time trajectory planning for UCAV air-to-surface attack using inverse dynamics optimization method and receding horizon control

This paper presents a computationally efficient real-time trajectory planning framework for typical unmanned combat aerial vehicle （UCAV） performing autonomous air-to-surface （A/S） attack. It combines the benefits of inverse dynamics optimization method and receding horizon optimal control technique. Firstly, the ground attack trajectory planning problem is mathematically formulated as a receding horizon optimal control problem （RHC-OCP）. In particular, an approximate elliptic launch acceptable region （LAR） model is proposed to model the critical weapon delivery constraints. Secondly, a planning algorithm based on inverse dynamics optimization, which has high computational efficiency and good convergence properties, is developed to solve the RHCOCP in real-time. Thirdly, in order to improve robustness and adaptivity in a dynamic and uncer- tain environment, a two-degree-of-freedom （2-DOF） receding horizon control architecture is introduced and a regular real-time update strategy is proposed as well, and the real-time feedback can be achieved and the not-converged situations can be handled. Finally, numerical simulations demon- strate the efficiency of this framework, and the results also show that the presented technique is well suited for real-time implementation in dynamic and uncertain environment.