HEURISTIC PARTICLE SWARM OPTIMIZATION ALGORITHM FOR AIR COMBAT DECISION-MAKING ON CMTA

Combining the heuristic algorithm （HA） developed based on the specific knowledge of the cooperative multiple target attack （CMTA） tactics and the particle swarm optimization （PSO）, a heuristic particle swarm optimization （HPSO） algorithm is proposed to solve the decision-making （DM） problem. HA facilitates to search the local optimum in the neighborhood of a solution, while the PSO algorithm tends to explore the search space for possible solutions. Combining the advantages of HA and PSO, HPSO algorithms can find out the global optimum quickly and efficiently. It obtains the DM solution by seeking for the optimal assignment of missiles of friendly fighter aircrafts （FAs） to hostile FAs. Simulation results show that the proposed algorithm is superior to the general PSO algorithm and two GA based algorithms in searching for the best solution to the DM problem.

Air combat decision-making of multiple UCAVs based on constraint strategy games

Game theory can be applied to the air combat decision-making problem of multiple unmanned combat air vehicles(UCAVs).However,it is difficult to have satisfactory decision-making results completely relying on air combat situation information,because there is a lot of time-sensitive information in a complex air combat environment.In this paper,a constraint strategy game approach is developed to generate intelligent decision-making for multiple UCAVs in complex air combat environment with air combat situation information and time-sensitive information.Initially,a constraint strategy game is employed to model attack-defense decision-making problem in complex air combat environment.Then,an algorithm is proposed for solving the constraint strategy game based on linear programming and linear inequality(CSG-LL).Finally,an example is given to illustrate the effectiveness of the proposed approach.

Autonomous air combat decision-making of UAV based on parallel self-play reinforcement learning

Aiming at addressing the problem of manoeuvring decision-making in UAV air combat,this study establishes a one-to-one air combat model,defines missile attack areas,and uses the non-deterministic policy Soft-Actor-Critic(SAC)algorithm in deep reinforcement learning to construct a decision model to realize the manoeuvring process.At the same time,the complexity of the proposed algorithm is calculated,and the stability of the closed-loop system of air combat decision-making controlled by neural network is analysed by the Lyapunov function.This study defines the UAV air combat process as a gaming process and proposes a Parallel Self-Play training SAC algorithm(PSP-SAC)to improve the generalisation performance of UAV control decisions.Simulation results have shown that the proposed algorithm can realize sample sharing and policy sharing in multiple combat environments and can significantly improve the generalisation ability of the model compared to independent training.

Approach to WTA in air combat using IAFSA-IHS algorithm

In this paper, a static weapon target assignment（WTA）problem is studied. As a critical problem in cooperative air combat,outcome of WTA directly influences the battle. Along with the cost of weapons rising rapidly, it is indispensable to design a target assignment model that can ensure minimizing targets survivability and weapons consumption simultaneously. Afterwards an algorithm named as improved artificial fish swarm algorithm-improved harmony search algorithm（IAFSA-IHS） is proposed to solve the problem. The effect of the proposed algorithm is demonstrated in numerical simulations, and results show that it performs positively in searching the optimal solution and solving the WTA problem.

UAV Maneuvering Decision-Making Algorithm Based on Twin Delayed Deep Deterministic Policy Gradient Algorithm

Aiming at intelligent decision-making of unmanned aerial vehicle(UAV)based on situation information in air combat,a novelmaneuvering decision method based on deep reinforcement learning is proposed in this paper.The autonomous maneuvering model ofUAV is established byMarkovDecision Process.The Twin DelayedDeep Deterministic Policy Gradient(TD3)algorithm and the Deep Deterministic Policy Gradient(DDPG)algorithm in deep reinforcement learning are used to train the model,and the experimental results of the two algorithms are analyzed and compared.The simulation experiment results show that compared with the DDPG algorithm,the TD3 algorithm has stronger decision-making performance and faster convergence speed and is more suitable for solving combat problems.The algorithm proposed in this paper enables UAVs to autonomously make maneuvering decisions based on situation information such as position,speed,and relative azimuth,adjust their actions to approach,and successfully strike the enemy,providing a new method for UAVs to make intelligent maneuvering decisions during air combat.

OPTIMIZATION FOR COMBAT CONFIGURATION OF AIR DEFENSE WEAPON SYSTEMS

At evaluating the combat effectiveness of the defense system, target′s probability to penetrate the defended area is a primary care taking index. In this paper, stochastic model to compete the probability that target penetrates the defended area along any flight path is established by the state analysis and statistical equilibrium analysis of stochastic service system theory. The simulated annealing algorithm is an enlightening random search method based on Monte Carlo recursion, and it can find global optimal solution by simulating annealing process. Combining stochastic model to compete the probability and simulated annealing algorithm, this paper establishes the method to solve problem quantitatively about combat configuration optimization of weapon systems. The calculated result shows that the perfect configuration for fire cells of the weapon is fast found by using this method, and this quantificational method for combat configuration is faster and more scientific than previous one based on principle via map fire field.

Cooperative decision-making algorithm with efficient convergence for UCAV formation in beyond-visual-range air combat based on multi-agent reinforcement learning

Highly intelligent Unmanned Combat Aerial Vehicle(UCAV)formation is expected to bring out strengths in Beyond-Visual-Range(BVR)air combat.Although Multi-Agent Reinforcement Learning(MARL)shows outstanding performance in cooperative decision-making,it is challenging for existing MARL algorithms to quickly converge to an optimal strategy for UCAV formation in BVR air combat where confrontation is complicated and reward is extremely sparse and delayed.Aiming to solve this problem,this paper proposes an Advantage Highlight Multi-Agent Proximal Policy Optimization(AHMAPPO)algorithm.First,at every step,the AHMAPPO records the degree to which the best formation exceeds the average of formations in parallel environments and carries out additional advantage sampling according to it.Then,the sampling result is introduced into the updating process of the actor network to improve its optimization efficiency.Finally,the simulation results reveal that compared with some state-of-the-art MARL algorithms,the AHMAPPO can obtain a more excellent strategy utilizing fewer sample episodes in the UCAV formation BVR air combat simulation environment built in this paper,which can reflect the critical features of BVR air combat.The AHMAPPO can significantly increase the convergence efficiency of the strategy for UCAV formation in BVR air combat,with a maximum increase of 81.5%relative to other algorithms.

UAV cooperative air combat maneuver decision based on multi-agent reinforcement learning

In order to improve the autonomous ability of unmanned aerial vehicles(UAV)to implement air combat mission,many artificial intelligence-based autonomous air combat maneuver decision-making studies have been carried out,but these studies are often aimed at individual decision-making in 1 v1 scenarios which rarely happen in actual air combat.Based on the research of the 1 v1 autonomous air combat maneuver decision,this paper builds a multi-UAV cooperative air combat maneuver decision model based on multi-agent reinforcement learning.Firstly,a bidirectional recurrent neural network(BRNN)is used to achieve communication between UAV individuals,and the multi-UAV cooperative air combat maneuver decision model under the actor-critic architecture is established.Secondly,through combining with target allocation and air combat situation assessment,the tactical goal of the formation is merged with the reinforcement learning goal of every UAV,and a cooperative tactical maneuver policy is generated.The simulation results prove that the multi-UAV cooperative air combat maneuver decision model established in this paper can obtain the cooperative maneuver policy through reinforcement learning,the cooperative maneuver policy can guide UAVs to obtain the overall situational advantage and defeat the opponents under tactical cooperation.

Target distribution in cooperative combat based on Bayesian optimization algorithm

Target distribution in cooperative combat is a difficult and emphases. We build up the optimization model according to the rule of fire distribution. We have researched on the optimization model with BOA. The BOA can estimate the joint probability distribution of the variables with Bayesian network, and the new candidate solutions also can be generated by the joint distribution. The simulation example verified that the method could be used to solve the complex question, the operation was quickly and the solution was best.

Shuffled frog leaping algorithm with non-dominated sorting for dynamic weapon-target assignment

The dynamic weapon target assignment(DWTA)problem is of great significance in modern air combat.However,DWTA is a highly complex constrained multi-objective combinatorial optimization problem.An improved elitist non-dominated sorting genetic algorithm-II(NSGA-II)called the non-dominated shuffled frog leaping algorithm(NSFLA)is proposed to maximize damage to enemy targets and minimize the self-threat in air combat constraints.In NSFLA,the shuffled frog leaping algorithm(SFLA)is introduced to NSGA-II to replace the inside evolutionary scheme of the genetic algorithm(GA),displaying low optimization speed and heterogeneous space search defects.Two improvements have also been raised to promote the internal optimization performance of SFLA.Firstly,the local evolution scheme,a novel crossover mechanism,ensures that each individual participates in updating instead of only the worst ones,which can expand the diversity of the population.Secondly,a discrete adaptive mutation algorithm based on the function change rate is applied to balance the global and local search.Finally,the scheme is verified in various air combat scenarios.The results show that the proposed NSFLA has apparent advantages in solution quality and efficiency,especially in many aircraft and the dynamic air combat environment.

基于改进支持向量回归的空战飞行动作识别

针对空战中飞机的飞行动作愈发复杂导致识别准确率低的问题,提出了改进支持向量回归的空战飞行动作识别方法,该方法采用高斯核函数作为线性核函数,利用混沌初始化和反向学习策略改进麻雀搜索算法,利用改进后的麻雀算法优化支持向量回归算法,具体表现为对支持向量回归算法中高斯核函数的参数进行优化,通过优化后的支持向量回归算法进行飞机动作识别。采用了五种基本的飞行动作和几种复杂的飞行动作验证该方法的识别准确率。仿真表明,优化后的支持向量回归算法与传统的支持向量回归算法、模糊支持向量机算法、传统聚类算法、神经网络算法相比,对基本飞行动作的平均识别率至少提升了2.2%,对复杂飞行动作的平均识别率至少提升了3.7%。

基于一种改进PPO算法的无人机空战自主机动决策方法研究

深度强化学习的应用为无人机自主机动决策提供了新的可能。提出一种基于态势评估模型重构与近端策略优化(PPO)算法相结合的无人机自主空战机动决策方法,为一对一近距空战提供了有效策略选择。首先,建立高保真六自由度无人机模型与近距空战攻击模型;其次,基于空战状态划分重构角度、速度、距离和高度态势函数,提出一种描述机动潜力的新型态势评估指标;之后,基于态势函数设计塑形奖励,并与基于规则的稀疏奖励、基于状态转换的子目标奖励共同构成算法奖励函数,增强了强化学习算法的引导能力;最后,设计专家系统作为对手,在高保真空战仿真平台(JSBSim)中对本文工作进行了评估。仿真验证,应用本文方法的智能体在对抗固定机动对手与专家系统对手时算法收敛速度与胜率都得到了有效提升。

空地异构无人系统侦察任务规划方法

相对空中同构无人系统,空地异构无人系统的运动能力、资源载荷、作战场景等异构性质会导致约束条件增多,使求解模型计算量显著增加,协同作战任务的建模和大规模问题的高效求解是需要解决的关键问题。以无人系统完成任务的时间、路径代价、侦察收益为目标函数,同时考虑无人平台续航能力等约束条件,合理构建了空地异构无人系统侦察任务的多目标规划模型;针对具有多威胁区的城市作战环境,考虑无人平台任务路径的安全性和时效性,分别提出了无人机和无人车改进A^(*)算法路径规划策略。针对蛇优化算法(snake optimizer,SO)优化效果不稳定、容易陷入局部最优解的问题,结合粒子群算法和遗传算法提出了改进蛇优化算法(improved snake optimizer,IMSO);通过Python语言进行了仿真验证和与现有算法的对比分析,验证了模型的可行性和算法的优越性。不同算法在由小到大的3种任务载荷设置下独求解10次,IMSO的平均目标函数值分别为SO的100.11%、108.99%和110.01%,可以看出IMSO能多次跳出局部最优,算法的稳定性、最终适应度值均好于SO,在较大规模问题的求解上更具有优越性。

基于有限忍耐度鸽群优化的无人机近距空战机动决策

由于对抗双方态势的快速变化,无人机近距空战机动自主决策困难且复杂,是空中对抗的一个难点。对此,提出一种基于有限忍耐度鸽群优化(FTPIO)算法的无人机近距空战机动决策方法。该方法主要包括基于机动动作库的对手行动预测和基于FTPIO算法的机动控制量和执行时间优化求解两个部分。为提升基本鸽群优化(PIO)算法的全局探索能力,引入有限忍耐度策略,在鸽子个体几次迭代中没有找到更优解时对其属性进行一次重置,避免陷入局部最优陷阱。该方法采用的优化变量是无人机运动模型控制变量的增量,打破了机动库的限制。通过和极小极大方法、基本PIO算法和粒子群优化(PSO)算法的仿真对抗测试结果表明,所提出的机动决策方法能够在近距空战中有效击败对手,产生更为灵活的欺骗性机动行为。

基于兵棋推演的空战编组对抗智能决策方法

基于兵棋研究的空战编组对抗方法主要使用规则或运筹等手段,存在假设不够合理、建模不准确、应变性差等缺陷。强化学习算法可以根据作战数据自主学习编组对抗策略,以应对复杂的战场情况,但现有强化学习对作战数据要求高,当动作空间过大时,算法收敛慢,且对仿真平台有较高的要求。针对上述问题,提出了一种融合知识数据和强化学习的空战编组对抗智能决策方法,该决策方法的输入是战场融合态势,使用分层决策框架控制算子选择并执行任务,上层包含使用专家知识驱动的动作选择器,下层包含使用专家知识和作战规则细化的避弹动作执行器、侦察动作执行器和使用强化学习算法控制的打击动作执行器。最后基于典型作战场景进行实验,验证了该方法的可行性和实用性,且具有建模准确、训练高效的优点。

Air Combat Assignment Problem Based on Bayesian Optimization Algorithm

In order to adapt to the changing battlefield situation and improve the combat effectiveness of air combat,the problem of air battle allocation based on Bayesian optimization algorithm(BOA)is studied.First,we discuss the number of fighters on both sides,and apply cluster analysis to divide our fighter into the same number of groups as the enemy.On this basis,we sort each of our fighters'different advantages to the enemy fighters,and obtain a series of target allocation schemes for enemy attacks by first in first serviced criteria.Finally,the maximum advantage function is used as the target,and the BOA is used to optimize the model.The simulation results show that the established model has certain decision-making ability,and the BOA can converge to the global optimal solution at a faster speed,which can effectively solve the air combat task assignment problem.

Cooperative Decision-Making for Multiple UAVs Autonomous Confrontation

This paper presents a rule-based framework for addressing decision-making problems within the context of the "UI-STRIVE"Competition.First,two distinct autonomous confrontation scenarios are described:autonomous air combat and cooperative interception.Second,a State-Event-Condition-Action(SECA)decision-making framework is developed,which integrates thefinite state machine and event-condition-action frameworks.This framework provides three products to describe rules,i.e.the SECA model,the SECA state chart,and the SECA rule description.Third,the situation assessment and target assignment during autonomous air combat are investigated,and the mathematical models are established.Finally,the decisionmaking model's rationality and feasibility are verified through data simulation and analysis.

HEURISTIC QUANTUM GENETIC ALGORITHM FOR AIR COMBAT DECISION MAKING ON COOPERATIVE MULTIPLE TARGET ATTACK

In order to achieve the optimal attack outcome in the air combat under the beyond visual range(BVR)condition,the decision-making(DM)problem which is to set a proper assignment for the friendly fighters on the hostile fighters is the most crucial task for cooperative multiple target attack(CMTA).In this paper,a heuristic quantum genetic algorithm(HQGA)is proposed to solve the DM problem.The originality of our work can be supported in the following aspects:(1)the HQGA assigns all hostile fighters to every missile rather than fighters so that the HQGA can encode chromosomes with quantum bits(Q-bits);(2)the relative successful sequence probability(RSSP)is defined,based on which the priority attack vector is constructed;(3)the HQGA can heuristically modify quantum chromosomes according to modification technique proposed in this paper;(4)last but not the least,in some special conditions,the HQGA gets rid of the constraint described by other algorithms that to obtain a better result.In the end of this paper,two examples are illustrated to show that the HQGA has its own advantage over other algorithms when dealing with the DM problem in the context of CMTA.

基于启发式蚁群算法的协同多目标攻击空战决策研究

协同多目标攻击空战决策是现代战机在超视距条件下进行协同空战的关键技术之一。它是寻求一个优化分配方案,将目标分配给各友机,力求使攻击效果最优。本文在对协同多目标攻击战术进行深入分析的基础上,提出了一种用于空战决策的启发式蚁群算法,该算法通过求解友机导弹对目标的最优分配来确定空战决策方案。仿真实验表明所提出的启发式蚁群算法对最优解的搜索效率明显优于基本蚁群算法,是一种求解协同多目标攻击空战决策问题的有效算法。

基于SAGA的协同多目标攻击决策

以超视距协同空战为背景,在对每个目标分配一枚导弹攻击的模式下,研究了协同多目标攻击空战决策问题.首先,基于对空战威胁态势的分析,将协同多目标攻击决策问题转化为导弹目标攻击分配的优化问题并建立其攻击效能评估模型.然后,提出将模拟退火遗传算法(SAGA)用于该问题的寻优,算法中个体采用整数编码,并采用非常规的交叉与变异操作产生新的个体.在进化结束后,通过最佳导弹目标分配个体求得最终协同攻击决策方案.仿真结果表明所提出的算法对最优分配方案的搜索效率明显优于单纯的遗传算法.