Cooperative maneuver decision making for multi-UAV air combat based on incomplete information dynamic game

Cooperative autonomous air combat of multiple unmanned aerial vehicles(UAVs)is one of the main combat modes in future air warfare,which becomes even more complicated with highly changeable situation and uncertain information of the opponents.As such,this paper presents a cooperative decision-making method based on incomplete information dynamic game to generate maneuver strategies for multiple UAVs in air combat.Firstly,a cooperative situation assessment model is presented to measure the overall combat situation.Secondly,an incomplete information dynamic game model is proposed to model the dynamic process of air combat,and a dynamic Bayesian network is designed to infer the tactical intention of the opponent.Then a reinforcement learning framework based on multiagent deep deterministic policy gradient is established to obtain the perfect Bayes-Nash equilibrium solution of the air combat game model.Finally,a series of simulations are conducted to verify the effectiveness of the proposed method,and the simulation results show effective synergies and cooperative tactics.

UAV cooperative air combat maneuver decision based on multi-agent reinforcement learning

In order to improve the autonomous ability of unmanned aerial vehicles(UAV)to implement air combat mission,many artificial intelligence-based autonomous air combat maneuver decision-making studies have been carried out,but these studies are often aimed at individual decision-making in 1 v1 scenarios which rarely happen in actual air combat.Based on the research of the 1 v1 autonomous air combat maneuver decision,this paper builds a multi-UAV cooperative air combat maneuver decision model based on multi-agent reinforcement learning.Firstly,a bidirectional recurrent neural network(BRNN)is used to achieve communication between UAV individuals,and the multi-UAV cooperative air combat maneuver decision model under the actor-critic architecture is established.Secondly,through combining with target allocation and air combat situation assessment,the tactical goal of the formation is merged with the reinforcement learning goal of every UAV,and a cooperative tactical maneuver policy is generated.The simulation results prove that the multi-UAV cooperative air combat maneuver decision model established in this paper can obtain the cooperative maneuver policy through reinforcement learning,the cooperative maneuver policy can guide UAVs to obtain the overall situational advantage and defeat the opponents under tactical cooperation.

Extracting Decision Rules for Cooperative Team Air Combat Based on Rough Set Theory

In order to reduce redundant features in air combat information and to meet the requirements of real-time decision in combat, rough set theory is introduced to the tactical decision analysis in cooperative team air combat. An algorithm of attribute reduction for extracting key combat information and generating tactical rules from given air combat databases is presented. Then, considering the practical requirements of team combat, a method for reduction of attribute-values under single decision attribute is extended to the reduction under multi-decision attributes. Finally, the algorithm is verified with an example for tactical choices in team air combat. The results show that, the redundant attributes in air combat information can be reduced, and that the main combat attributes, i.e., the information about radar command and medium-range guided missile, can be obtained with the algorithm mentioned above, moreover, the minimal reduced strategy for tactical decision can be generated without losing the result of key information classification. The decision rules extracted agree with the real situation of team air combat.

A Multi-UCAV cooperative occupation method based on weapon engagement zones for beyond-visual-range air combat

Recent advances in on-board radar and missile capabilities,combined with individual payload limitations,have led to increased interest in the use of unmanned combat aerial vehicles(UCAVs)for cooperative occupation during beyond-visual-range(BVR)air combat.However,prior research on occupational decision-making in BVR air combat has mostly been limited to one-on-one scenarios.As such,this study presents a practical cooperative occupation decision-making methodology for use with multiple UCAVs.The weapon engagement zone(WEZ)and combat geometry were first used to develop an advantage function for situational assessment of one-on-one engagement.An encircling advantage function was then designed to represent the cooperation of UCAVs,thereby establishing a cooperative occupation model.The corresponding objective function was derived from the one-on-one engagement advantage function and the encircling advantage function.The resulting model exhibited similarities to a mixed-integer nonlinear programming(MINLP)problem.As such,an improved discrete particle swarm optimization(DPSO)algorithm was used to identify a solution.The occupation process was then converted into a formation switching task as part of the cooperative occupation model.A series of simulations were conducted to verify occupational solutions in varying situations,including two-on-two engagement.Simulated results showed these solutions varied with initial conditions and weighting coefficients.This occupation process,based on formation switching,effectively demonstrates the viability of the proposed technique.These cooperative occupation results could provide a theoretical framework for subsequent research in cooperative BVR air combat.

Task assignment under constraint of timing sequential for cooperative air combat

According to the previous achievement, the task assignment under the constraint of timing continuity for a cooperative air combat is studied. An extensive task assignment scenario with the background of the cooperative air combat is proposed. The utility and time of executing a task as well as the continuous combat ability are defined. The concept of the matching method of weapon and target is modified based on the analysis of the air combat scenario. The constraint framework is also redefined according to a new objective function. The constraints of timing and continuity are formulated with a new method, at the same time, the task assignment and integer programming models of the cooperative combat are established. Finally, the assignment problem is solved using the integrated linear programming software and the simulation shows that it is feasible to apply this modified model in the cooperative air combat for tasks cooperation and it is also efficient to optimize the resource assignment.

Target distribution in cooperative combat based on Bayesian optimization algorithm

Target distribution in cooperative combat is a difficult and emphases. We build up the optimization model according to the rule of fire distribution. We have researched on the optimization model with BOA. The BOA can estimate the joint probability distribution of the variables with Bayesian network, and the new candidate solutions also can be generated by the joint distribution. The simulation example verified that the method could be used to solve the complex question, the operation was quickly and the solution was best.

HEURISTIC PARTICLE SWARM OPTIMIZATION ALGORITHM FOR AIR COMBAT DECISION-MAKING ON CMTA

Combining the heuristic algorithm （HA） developed based on the specific knowledge of the cooperative multiple target attack （CMTA） tactics and the particle swarm optimization （PSO）, a heuristic particle swarm optimization （HPSO） algorithm is proposed to solve the decision-making （DM） problem. HA facilitates to search the local optimum in the neighborhood of a solution, while the PSO algorithm tends to explore the search space for possible solutions. Combining the advantages of HA and PSO, HPSO algorithms can find out the global optimum quickly and efficiently. It obtains the DM solution by seeking for the optimal assignment of missiles of friendly fighter aircrafts （FAs） to hostile FAs. Simulation results show that the proposed algorithm is superior to the general PSO algorithm and two GA based algorithms in searching for the best solution to the DM problem.

Cooperative Path Dynamic Planning Model of UCAV Team Based on Global Optimization Method

Cooperative path dynamic planning of a UCAV （unmanned combat air vehicle） team not only considers the capability of task requirement of single UCAV, but also considers the cooperative dynamic connection among members of the UCAV team. A cooperative path dynamic planning model of the UCAV team by applying a global optimization method is discussed in this paper and the corresponding model is built and analyzed. By the example simulation, the reasonable result acquired indicates that the model could meet dynamic planning demand under the circumstance of membership functions. The model is easy to be realized and has good practicability.

有人/无人机协同空战研究进展综述

有人/无人机协同空战是未来空战的重要发展方向,有人机和无人机协同运用、优势互补,形成灵活高效的作战体系,将对未来空战效能的提升发挥巨大作用。首先,阐述了近年来美军作战概念的发展演进,分析了美军有人/无人机协同作战的相关项目发展,提炼总结了有人/无人机协同空战的特点;其次,以穿透性制空作战为场景,分析了有人/无人机协同空战的具体运用和作战流程;最后,从航迹规划、空战决策、编队控制3个方面对有人/无人机协同空战的关键技术和研究现状进行梳理和分析,并对未来有人/无人机协同空战的研究发展方向进行了总结和展望。

有人/无人机协同空战任务规划技术

随着“分布式作战”“马赛克战”等新型作战概念的提出,无人机作为新型空战力量,依靠其灵活性、机动性、互操作性和低廉的成本,有效地弥补了有人机在空战中的作战“盲区”,将其纳入现代空战的任务规划体系,与有人机协同配合执行空战任务,会对空战结果产生巨大的影响。研究了基于数据的空战离线任务规划技术和基于任务的空战在线任务规划技术,并基于OODA环对有人/无人机协同空战任务规划技术进行了综述,结合该技术在测试进程和实际作战应用中存在的不足,展望了该技术未来的发展趋势,对下一步开展有人/无人机协同空战任务规划研究提供借鉴参考。

Cooperative Threat Assessment of Multi-aircrafts Based on Synthetic Fuzzy Cognitive Map

Threat assessment is one of the most important parts of the tactical decisions,and it has a very important influence on task allocation.An application of fuzzy cognitive map(FCM) for target threat assessment in the air combat is introduced.Considering the fact that the aircrafts participated in the cooperation may not have the same threat assessment mechanism,two different FCM models are established.Using the method of combination,the model of cooperative threat assessment in air combat of multi-aircrafts is established.Simulation results show preliminarily that the method is reasonable and effective.Using FCM for threat assessment is feasible.

HEURISTIC QUANTUM GENETIC ALGORITHM FOR AIR COMBAT DECISION MAKING ON COOPERATIVE MULTIPLE TARGET ATTACK

In order to achieve the optimal attack outcome in the air combat under the beyond visual range(BVR)condition,the decision-making(DM)problem which is to set a proper assignment for the friendly fighters on the hostile fighters is the most crucial task for cooperative multiple target attack(CMTA).In this paper,a heuristic quantum genetic algorithm(HQGA)is proposed to solve the DM problem.The originality of our work can be supported in the following aspects:(1)the HQGA assigns all hostile fighters to every missile rather than fighters so that the HQGA can encode chromosomes with quantum bits(Q-bits);(2)the relative successful sequence probability(RSSP)is defined,based on which the priority attack vector is constructed;(3)the HQGA can heuristically modify quantum chromosomes according to modification technique proposed in this paper;(4)last but not the least,in some special conditions,the HQGA gets rid of the constraint described by other algorithms that to obtain a better result.In the end of this paper,two examples are illustrated to show that the HQGA has its own advantage over other algorithms when dealing with the DM problem in the context of CMTA.

多无人机协同空战任务规划仿真系统

面向多无人机协同空战任务规划方法性能验证的需求,开发了一种基于VR-Forces仿真引擎的多无人机协同空战任务规划仿真系统,对该仿真系统的总体架构、红方子系统、蓝方子系统、白方子系统和强化学习算法训练子系统进行了设计。对算法库、强化学习算法训练和人机交互等关键技术提出了针对性的解决方案,提升了该仿真系统的稳定性、可扩展性和功能性。利用该仿真系统对多无人机协同空战目标分配、重决策算法等典型任务规划方法进行了测试和仿真实验。结果表明,所开发的仿真系统可以很好地支持多无人机协同空战任务规划方法的测试和验证。

非完备信息下的超视距空战双机协同战术识别

针对超视距(BVR)空战过程中,受探测装置性能限制和敌方干扰等原因,导致目标信息易缺失,从而难以实时准确地识别敌方协同空战战术的问题,提出了一种基于动态贝叶斯网络(DBN)与参数学习的超视距空战双机协同战术识别方法。分析了超视距空战条件下的双机协同战术特征,根据长机和僚机的职能分工、当前态势及机动动作,构建了识别网络模型;为提高模型对双机协同战术的识别概率,采用期望最大参数学习方法优化网络参数;基于自回归模型对缺失目标信息进行修补,提出非完备信息下的双机协同战术识别推理算法。通过开展空战对抗仿真实验,验证了双机协同战术识别方法对于非完备信息下的超视距空战双机协同战术具有较高的识别概率和较好的实时性。

有人—无人机协同空战机动决策研究

目前,有关无人机空战的研究主要考虑无人机的完全自主决策机动算法,关于有人机有限监督决策下的空战机动决策的研究鲜有报道,更缺乏对有人—无人机协同作战的研究。为实现无人机协同空战过程中的自主机动,设计一种基于路径规划技术的有人—无人机协同空战机动决策模型。首先,引入动态栅格环境,自适应调整栅格规模和分辨率,以弥补静态栅格环境规划空间越大规划效率越低的缺陷;然后,将A star算法规划路径作为参考路径,提出ACO-A star混合路径规划算法,以提升ACO算法的寻优效能;最后,基于均值聚类算法设计有人—无人机协同空战机动决策算法。进行空战对抗仿真模拟,结果表明:所提出的算法具有更好的决策正确性,可有效提升空战胜率。

基于近端策略优化的空战决策算法研究

面对未来有/无人机协同作战场景,实时准确的空战决策是制胜的关键。复杂的空中环境、瞬变的态势数据以及多重繁琐的作战任务,使有/无人机协同作战将替代单机作战成为未来空战的发展趋势,但多智能体建模和训练过程却面临奖励分配困难、网络难收敛的问题。针对5v5有/无人机协同的空战场景,抽象出有人机和无人机智能体的特征模型,提出基于近端策略优化算法的空战智能决策算法,通过设置态势评估奖励引导空战过程中有/无人机智能体的决策行为向有利态势发展,实现在与环境的实时交互中,输出空战决策序列。通过仿真实验对所提空战决策算法进行验证,结果表明:本文提出的算法在经过训练学习后,能够适应复杂的战场态势,在连续动作空间中得到稳定合理的决策策略。

基于改进合同网的分布式协同目标分配

针对传统集中式目标分配算法存在的抗毁能力弱、可扩展性差等问题,提出一种基于改进合同网协议的完全分布式多平台协同目标分配算法。根据作战环境约束、任务约束以及火力平台自身性能约束等一系列约束条件构建目标分配模型;在目标分配的过程中增加合同交换操作,提升分配收益。结果表明:该算法能在提高整体收益的同时,减少目标分配反应时间,具备较好可扩展性。

水面舰艇编队软硬武器协同防空反导作战研究

在对空防御作战中最常用的方式就是综合运用有源干扰、箔条干扰与舰空导弹武器系统结合进行协同防空反导,论文探索和研究单平台和编队软硬武器作战应用规则,并通过单舰针对性训练和舰艇编队防空反导训练进行检验。训练结果检验了软硬武器协同作战应用规则的有效性,提高了舰艇编队协同防空能力,具有一定的军事效益,同时软武器的使用降低了导弹的使用成本,具备一定的经济效益,并为舰艇编队其它软硬武器综合运用提供借鉴。

基于DODAF作战视图的空地协同作战模型设计

空地协同作战是一种跨平台协同联合防空的典型作战模式,以美国防部架构框架(departmentof defense architecture framework,DODAF)作战视图为研究工具,构建高层作战概念、组织关系、作战活动、作战资源等空地协同作战视图模型,相较于传统的作战过程设计,较为直观地描述了空地协同作战过程和指挥控制流程等,通过构建作战模型以及信息交互、作战逻辑关系自洽验证,对于后续优化空地协同作战体系、顶层设计和装备研制具有一定的参考意义。

动态协同空战攻击决策问题研究

攻击决策问题是协同空战领域研究的核心问题之一,通过优化战机武器资源配置,以提高战机协同作战效能。针对动态协同空战攻击决策问题,系统分析空战环境的动态性,将空战动态划分为连续动态和离散动态,并建立多约束条件下动态协同空战攻击决策优化模型;设计了空战环境变化监测策略和种群多样性策略,通过迭代窗的设置,基于离散粒子群算法,提出了一种基于滚动迭代窗的启发式变异离散粒子群算法的动态协同攻击决策方法。通过仿真实验验证了所提方法能够在连续和离散动态条件下合理地规划协同空战攻击决策。