Mastering air combat game with deep reinforcement learning

Reinforcement learning has been applied to air combat problems in recent years,and the idea of curriculum learning is often used for reinforcement learning,but traditional curriculum learning suffers from the problem of plasticity loss in neural networks.Plasticity loss is the difficulty of learning new knowledge after the network has converged.To this end,we propose a motivational curriculum learning distributed proximal policy optimization(MCLDPPO)algorithm,through which trained agents can significantly outperform the predictive game tree and mainstream reinforcement learning methods.The motivational curriculum learning is designed to help the agent gradually improve its combat ability by observing the agent's unsatisfactory performance and providing appropriate rewards as a guide.Furthermore,a complete tactical maneuver is encapsulated based on the existing air combat knowledge,and through the flexible use of these maneuvers,some tactics beyond human knowledge can be realized.In addition,we designed an interruption mechanism for the agent to increase the frequency of decisionmaking when the agent faces an emergency.When the number of threats received by the agent changes,the current action is interrupted in order to reacquire observations and make decisions again.Using the interruption mechanism can significantly improve the performance of the agent.To simulate actual air combat better,we use digital twin technology to simulate real air battles and propose a parallel battlefield mechanism that can run multiple simulation environments simultaneously,effectively improving data throughput.The experimental results demonstrate that the agent can fully utilize the situational information to make reasonable decisions and provide tactical adaptation in the air combat,verifying the effectiveness of the algorithmic framework proposed in this paper.

A function-based behavioral modeling method for air combat simulation

Today’s air combat has reached a high level of uncertainty where continuous or discrete variables with crisp values cannot be properly represented using fuzzy sets. With a set of membership functions, fuzzy logic is well-suited to tackle such complex states and actions. However, it is not necessary to fuzzify the variables that have definite discrete semantics.Hence, the aim of this study is to improve the level of model abstraction by proposing multiple levels of cascaded hierarchical structures from the perspective of function, namely, the functional decision tree. This method is developed to represent behavioral modeling of air combat systems, and its metamodel,execution mechanism, and code generation can provide a sound basis for function-based behavioral modeling. As a proof of concept, an air combat simulation is developed to validate this method and the results show that the fighter Alpha built using the proposed framework provides better performance than that using default scripts.

Optimal confrontation position selecting games model and its application to one-on-one air combat

In the air combat process,confrontation position is the critical factor to determine the confrontation situation,attack effect and escape probability of UAVs.Therefore,selecting the optimal confrontation position becomes the primary goal of maneuver decision-making.By taking the position as the UAV’s maneuver strategy,this paper constructs the optimal confrontation position selecting games(OCPSGs)model.In the OCPSGs model,the payoff function of each UAV is defined by the difference between the comprehensive advantages of both sides,and the strategy space of each UAV at every step is defined by its accessible space determined by the maneuverability.Then we design the limit approximation of mixed strategy Nash equilibrium(LAMSNQ)algorithm,which provides a method to determine the optimal probability distribution of positions in the strategy space.In the simulation phase,we assume the motions on three directions are independent and the strategy space is a cuboid to simplify the model.Several simulations are performed to verify the feasibility,effectiveness and stability of the algorithm.

Combating Cholera

Lying in her makeshift hospital bed,Joyce Tembo thanked medical personnel for evacuating her to the designated national cholera treatment centre,6 km north of Zambia’s capital Lusaka.She was recently diagnosed with diarrhoeal disease.Tembo,43,commended the medical sta!stationed at the treatment centre for their great service to thousands of patients,especially women and children seeking urgent treatment.“I am very grateful to the Chinese doctors who attended to me as soon as the ambulance rushed me to the clinic where I received urgent treatment;they have really saved my life,”Tembo told ChinAfrica.But not all residents in her community are as lucky as her.Many in the densely populated slums die every day due to the area’s poor sanitation-one of the major causes of the cholera outbreak.

Immune multi-agent model using vaccine for cooperative air-defense system of systems for surface warship formation based on danger theory

Aiming at the problem on cooperative air-defense of surface warship formation, this paper maps the cooperative airdefense system of systems （SoS） for surface warship formation （CASoSSWF） to the biological immune system （BIS） according to the similarity of the defense mechanism and characteristics between the CASoSSWF and the BIS, and then designs the models of components and the architecture for a monitoring agent, a regulating agent, a killer agent, a pre-warning agent and a communicating agent by making use of the theories and methods of the artificial immune system, the multi-agent system （MAS）, the vaccine and the danger theory （DT）. Moreover a new immune multi-agent model using vaccine based on DT （IMMUVBDT） for the cooperative air-defense SoS is advanced. The immune response and immune mechanism of the CASoSSWF are analyzed. The model has a capability of memory, evolution, commendable dynamic environment adaptability and self-learning, and embodies adequately the cooperative air-defense mechanism for the CASoSSWF. Therefore it shows a novel idea for the CASoSSWF which can provide conception models for a surface warship formation operation simulation system.

Autonomous air combat decision-making of UAV based on parallel self-play reinforcement learning

Aiming at addressing the problem of manoeuvring decision-making in UAV air combat,this study establishes a one-to-one air combat model,defines missile attack areas,and uses the non-deterministic policy Soft-Actor-Critic(SAC)algorithm in deep reinforcement learning to construct a decision model to realize the manoeuvring process.At the same time,the complexity of the proposed algorithm is calculated,and the stability of the closed-loop system of air combat decision-making controlled by neural network is analysed by the Lyapunov function.This study defines the UAV air combat process as a gaming process and proposes a Parallel Self-Play training SAC algorithm(PSP-SAC)to improve the generalisation performance of UAV control decisions.Simulation results have shown that the proposed algorithm can realize sample sharing and policy sharing in multiple combat environments and can significantly improve the generalisation ability of the model compared to independent training.

Cooperative maneuver decision making for multi-UAV air combat based on incomplete information dynamic game

Cooperative autonomous air combat of multiple unmanned aerial vehicles(UAVs)is one of the main combat modes in future air warfare,which becomes even more complicated with highly changeable situation and uncertain information of the opponents.As such,this paper presents a cooperative decision-making method based on incomplete information dynamic game to generate maneuver strategies for multiple UAVs in air combat.Firstly,a cooperative situation assessment model is presented to measure the overall combat situation.Secondly,an incomplete information dynamic game model is proposed to model the dynamic process of air combat,and a dynamic Bayesian network is designed to infer the tactical intention of the opponent.Then a reinforcement learning framework based on multiagent deep deterministic policy gradient is established to obtain the perfect Bayes-Nash equilibrium solution of the air combat game model.Finally,a series of simulations are conducted to verify the effectiveness of the proposed method,and the simulation results show effective synergies and cooperative tactics.

The personal protective equipment(PPE)based on individual combat:A systematic review and trend analysis

With the development of ordnance technology,the survival and safety of individual combatants in hightech warfare are under serious threat,and the Personal Protective Equipment(PPE),as an important guarantee to reduce casualties and maintain military combat effectiveness,is widely developed.This paper systematically reviewed various PPE based on individual combat through literature research and comprehensive discussion,and introduced in detail the latest application progress of PPE in terms of material and technology from three aspects:individual integrated protection system,traditional protection equipment,and intelligent protection equipment,respectively,and discussed in depth the functional improvement and optimization status brought by advanced technology for PPE,focusing on the achievements of individual equipment technology application.Finally,the problems and technical bottlenecks in the development of PPE were analyzed and summarized,and the development trend of PPE were pointed out.The results of the review will provide a forward-looking reference for the current development of individual PPE,and are important guidance for the design and technological innovation of advanced equipment based on the future technological battlefield.

Entry-Level Forward Surgical Team Training Is Associated with Increased Confidence of Primary Combat Surgeons

Background: In recent years, we have established an entry-level Forward Surgical Team (FST) training program in a Chinese military medical university for the 5th grade undergraduates, who would be deployed to different military medical services as primary combat surgeons. This study aimed to assess the role of this pre-service training in improving their confidence with combat medical skills, after several years since they received the training. Methods: We conducted a nationwide survey of 239 primary combat surgeons who have ever participated in an entry-level FST training program before deployment between June 2016 and June 2020, which was for evaluating on a 5-point Likert scale the benefits of entry-level FST training and conventional surgery training in improving their confidence with combat medical skills. The difference in scores was compared using the student t-test. Significance was considered as P Results: The total score was significantly higher for entry-level FST training than that for conventional surgery training (30.76 ± 4.33 vs. 28.95 ± 4.80, P There was no significant difference between the training for surgical skills confidence scores (18.03 ± 8.04 vs. 17.51 ± 8.30, P = 0.098), but for non-technical skills, the score of entry-level FST training was significantly higher than that of conventional surgery training (12.73 ± 5.39 vs. 11.44 ± 5.62, P The distributions of confidence scores were different under various subgroups by demographics. There were no significant differences in scores between the two training in all specific surgical skill sets except “life-saving surgery” (P = 0.011). Scores of all 4 non-technical skill sets were significantly higher for entry-level FST than those for conventional surgery training (P Conclusions: The training should be considered as an essential strategy to improve confidence in combat medical skills, especially life-saving surgery and non-technical skills, for primary combat surgeons.

基于“马赛克战”的水下有人-无人作战使用探讨

通过对“马赛克战”概念的背景和特点进行分析,提出基于马赛克的水下攻防作战运用构想,构建基于马赛克的水下有人-无人作战网络,分析多域背景下水下OODA作战网络的构成,利用OODA作战环理论对水下马赛克作战网络作战能力进行分析,并提出水下马赛克作战网络对抗单个平台和对抗作战网络的使用方法。最后,从通信组网、集群使用、人工智能、察打一体等4个方面对“马赛克战”概念下水下无人航行器装备使用提出建议。

联合全域作战态势下对战伤救治能力建设需求的思考

21世纪初,以人工智能为代表的颠覆性技术的突破与创新深刻影响了人类社会的发展,也成为引领全球新军事变革的核心组成部分。挽救伤员生命与技术创新是军事医学发展的原动力。如何在新的作战样式和态势下成功救治伤病员,是军事医学必须回答的问题。联合全域作战是一种全新的军事理念,代表着未来作战样式的发展方向,对未来战争具有很强的战略指导意义,也对战伤救治提出了更高的要求。“白金10分钟,黄金1小时”是战场一线救治时效的根本法则,战伤救治必须服务于联合全域作战需求与勤务要求。本文就联合全域作战态势下战伤救治能力建设的需求问题进行初步探讨。

基于强化学习的对抗意图识别

未来智能化战争复杂多变,敌我双方往往以对抗博弈情况出现,当我方作为攻击者时,如何有效隐藏我方意图实施有效打击,以及我方作为防御者时如何识别敌方意图实施有效防御,对战争局势的走向起到决定性作用。为把握战争局势,做到“知己知彼,百战不殆”,基于强化学习方法针对对抗意图识别问题提出了解决方案。仿真结果表明,基于强化学习被识别者可以有效地进行意图欺骗,而识别者可以通过意图识别设计方法有效地迫使被识别者暴露意图,从而采取攻防措施。

无人空战指控建模仿真方法研究

针对当前无人空战仿真中,缺乏支持全流程作战指挥控制建模与仿真这一问题,结合当前无人空战仿真系统相关研究,系统梳理和研究指控模型构建方法,通过探索不同仿真层级不同类型的无人机指挥控制建模原理和方法,设计并且构建了一套适合指挥信息系统装备特点的指控模型仿真系统。通过进行符合预设约束的想定设定,对目标作战任务进行了效能评估和对比,可为无人空战相关指挥信息系统装备的论证、评估方式提供理论和现实指导。

基于改进蚁群算法和共用航段的搜索救援通道规划

针对战斗搜救任务场景中,空中搜索救援等待区至跳伞区之间的往返路径规划问题,提出一种基于改进蚁群算法和共用航段的搜索救援通道规划方法。首先,针对基本蚁群算法易出现局部极值及收敛速度慢等缺点,对启发函数、状态转移策略、信息素更新策略进行了改进。然后,针对单起点多目标点的路径规划问题,提出了设置分航点以生成共用航段的搜索救援通道规划方法,并以总路径长度表征任务准备负荷量与分别规划航路进行对比。实验结果表明,在本文测试环境下,改进蚁群算法规划的路径长度分别缩减15.4%和14.2%,搜索救援通道的任务准备负荷量减少25.4%。所提出的搜索救援通道规划方法有一定的理论和应用价值。

无人机自主作战效能评估技术研究综述

无人机自主作战效能评估是衡量无人机在特定作战条件下能否运用自主能力完成预定目标任务的有效途径,是无人机自主作战技术论证和装备体系建设发展的核心问题。从无人机作战效能基本问题出发,首先,剖析无人机自主性与自主能力以及自主作战能力与自主作战效能的基本概念,给出无人机自主作战效能评估的基本流程;其次,分别从任务视角和原理角度对无人装备作战效能评估指标体系、评估方法的研究现状进行综述,归纳总结3类指标体系构建的现状和4类效能评估方法进展;最后,梳理当前评估中存在的问题,展望无人机效能评估技术的发展趋势。相关研究可为后续无人机自主作战效能评估提供技术参考和方向指导,具有重要的军事意义。

基于MBSE的作战能力跨域描述元建模方法

针对作战能力跨域描述存在语义二义性、不利于交流等问题,提出一种基于模型系统工程的元建模方法。抽象出作战能力元模型并给出形式化定义;从能力、作战和信息视角扩展作战能力元模型定义作战能力描述语言;通过UML Profile技术实现作战能力描述语言建模工具;借鉴面向对象分析方法思路提出作战能力描述语言建模方法。以美军“分布式防御”概念为例对提出方法进行实例验证。结果表明,提出的方法易于理解、可靠性和可扩展性强,有效消除语义二义性的同时满足多视角跨域描述作战能力的需求。

新质公安战斗力的生成逻辑、科学内涵与实践路径

新质公安战斗力的概念是顺应新质生产力的概念而提出的。新质生产力发展带来社会新发展格局,也同时带来社会治安新常态,公安机关为适应这一社会治安新常态需要努力构建新安全格局,并从加快形成和提升新质公安战斗力方面着手进行实践。加快形成和提升新质公安战斗力需要从关注其生成逻辑作为起点,从马克思主义生产力理论来把握其理论逻辑,从公安工作的历史发展经验来分析其历史逻辑,从支撑与服务中国式现代化的必然要求来认识其现实逻辑。新质公安战斗力的科学内涵需系统辩证地认识到“新”是工作起点,“质”是目标追求,“以新促质”是核心要义,而“战斗力”则是关键落脚点。新质公安战斗力的实践路径需要从思想政治、社会治理、队伍建设、创新发展等重要能力维度出发,为奋力推进公安工作现代化提供动力,从而支撑与服务中国式现代化。

新质公安战斗力的现实背景、理论解构与质态提升

新质公安战斗力是中国特色公安学学术话语体系的新范畴,它的提出是在推进中国式现代化过程中,立足国际国内两个大局,统筹实现高质量发展和高水平安全良性互动,将公安队伍建设置于党和国家发展大局中思考谋划而提出的重要概念,意味着公安核心战斗力质的变化和跃迁,具有动态发展的性质。新质公安战斗力科学客观地体现了公安工作现代化面对的政治方向之新、形势任务之新、构建理念之新、内涵要义之新、核心要素以及使命愿景之新。加快形成和提升新质公安战斗力,要高站位强化政治建设,宽视野运用系统观念,大力度推进改革创新,深层次提升科技赋能,全方位抓好人才队伍。

新形势下航天装备环境试验技术的未来发展趋势

从航天装备面临的实战化考核、数字化建设和低成本控制等新形势与新要求出发,梳理了环境试验技术在航天装备试验鉴定中存在的实战化考核能力不足、全寿命周期环境效应考核不充分等技术难题,结合航天装备环境试验的国内外发展现状,系统分析了航天装备环境试验技术的未来发展趋势,试验环境综合化、试验设计一体化、试验手段数字化在提升航天装备环境试验水平、缩短装备试验周期、降低试验成本方面具有显著优势,也是航天装备环境试验未来发展的重要方向。

美军军事仿真LVC系统发展与启示

针对当前多军兵种联合作战实战化训练的需求,建模和仿真领域的未来目标是创建统一的真实-虚拟-构造(LVC)集成架构,支持快速集成模型和开展仿真,用于联合作战训练、战术协同、拟制作战计划和评估等。通过探究LVC概念内涵和应用领域,分析美军LVC训练网络发展计划,梳理了分布式任务作战、联合仿真环境和美海军模拟训练三个典型LVC系统应用范例。借鉴美军应用LVC解决联合训练的经验做法,给出加快构建我军一体化联合训练的启示,为我军联合训练环境建设提供有价值的参考。