Mastering air combat game with deep reinforcement learning

Reinforcement learning has been applied to air combat problems in recent years,and the idea of curriculum learning is often used for reinforcement learning,but traditional curriculum learning suffers from the problem of plasticity loss in neural networks.Plasticity loss is the difficulty of learning new knowledge after the network has converged.To this end,we propose a motivational curriculum learning distributed proximal policy optimization(MCLDPPO)algorithm,through which trained agents can significantly outperform the predictive game tree and mainstream reinforcement learning methods.The motivational curriculum learning is designed to help the agent gradually improve its combat ability by observing the agent's unsatisfactory performance and providing appropriate rewards as a guide.Furthermore,a complete tactical maneuver is encapsulated based on the existing air combat knowledge,and through the flexible use of these maneuvers,some tactics beyond human knowledge can be realized.In addition,we designed an interruption mechanism for the agent to increase the frequency of decisionmaking when the agent faces an emergency.When the number of threats received by the agent changes,the current action is interrupted in order to reacquire observations and make decisions again.Using the interruption mechanism can significantly improve the performance of the agent.To simulate actual air combat better,we use digital twin technology to simulate real air battles and propose a parallel battlefield mechanism that can run multiple simulation environments simultaneously,effectively improving data throughput.The experimental results demonstrate that the agent can fully utilize the situational information to make reasonable decisions and provide tactical adaptation in the air combat,verifying the effectiveness of the algorithmic framework proposed in this paper.

A function-based behavioral modeling method for air combat simulation

ion by proposing multiple levels of cascaded hierarchi-cal structures from the perspective of function,namely,the func-tional decision tree.This method is developed to represent behavioral modeling of air combat systems,and its metamodel,execution mechanism,and code generation can provide a sound basis for function-based behavioral modeling.As a proof of con-cept,an air combat simulation is developed to validate this method and the results show that the fighter Alpha built using the proposed framework provides better performance than that using default scripts.

Optimal confrontation position selecting games model and its application to one-on-one air combat

In the air combat process,confrontation position is the critical factor to determine the confrontation situation,attack effect and escape probability of UAVs.Therefore,selecting the optimal confrontation position becomes the primary goal of maneuver decision-making.By taking the position as the UAV’s maneuver strategy,this paper constructs the optimal confrontation position selecting games(OCPSGs)model.In the OCPSGs model,the payoff function of each UAV is defined by the difference between the comprehensive advantages of both sides,and the strategy space of each UAV at every step is defined by its accessible space determined by the maneuverability.Then we design the limit approximation of mixed strategy Nash equilibrium(LAMSNQ)algorithm,which provides a method to determine the optimal probability distribution of positions in the strategy space.In the simulation phase,we assume the motions on three directions are independent and the strategy space is a cuboid to simplify the model.Several simulations are performed to verify the feasibility,effectiveness and stability of the algorithm.

Combating Cholera

Lying in her makeshift hospital bed,Joyce Tembo thanked medical personnel for evacuating her to the designated national cholera treatment centre,6 km north of Zambia’s capital Lusaka.She was recently diagnosed with diarrhoeal disease.Tembo,43,commended the medical sta!stationed at the treatment centre for their great service to thousands of patients,especially women and children seeking urgent treatment.“I am very grateful to the Chinese doctors who attended to me as soon as the ambulance rushed me to the clinic where I received urgent treatment;they have really saved my life,”Tembo told ChinAfrica.But not all residents in her community are as lucky as her.Many in the densely populated slums die every day due to the area’s poor sanitation-one of the major causes of the cholera outbreak.

Cooperative maneuver decision making for multi-UAV air combat based on incomplete information dynamic game

Cooperative autonomous air combat of multiple unmanned aerial vehicles(UAVs)is one of the main combat modes in future air warfare,which becomes even more complicated with highly changeable situation and uncertain information of the opponents.As such,this paper presents a cooperative decision-making method based on incomplete information dynamic game to generate maneuver strategies for multiple UAVs in air combat.Firstly,a cooperative situation assessment model is presented to measure the overall combat situation.Secondly,an incomplete information dynamic game model is proposed to model the dynamic process of air combat,and a dynamic Bayesian network is designed to infer the tactical intention of the opponent.Then a reinforcement learning framework based on multiagent deep deterministic policy gradient is established to obtain the perfect Bayes-Nash equilibrium solution of the air combat game model.Finally,a series of simulations are conducted to verify the effectiveness of the proposed method,and the simulation results show effective synergies and cooperative tactics.

Autonomous air combat decision-making of UAV based on parallel self-play reinforcement learning

Aiming at addressing the problem of manoeuvring decision-making in UAV air combat,this study establishes a one-to-one air combat model,defines missile attack areas,and uses the non-deterministic policy Soft-Actor-Critic(SAC)algorithm in deep reinforcement learning to construct a decision model to realize the manoeuvring process.At the same time,the complexity of the proposed algorithm is calculated,and the stability of the closed-loop system of air combat decision-making controlled by neural network is analysed by the Lyapunov function.This study defines the UAV air combat process as a gaming process and proposes a Parallel Self-Play training SAC algorithm(PSP-SAC)to improve the generalisation performance of UAV control decisions.Simulation results have shown that the proposed algorithm can realize sample sharing and policy sharing in multiple combat environments and can significantly improve the generalisation ability of the model compared to independent training.

The personal protective equipment(PPE)based on individual combat:A systematic review and trend analysis

With the development of ordnance technology,the survival and safety of individual combatants in hightech warfare are under serious threat,and the Personal Protective Equipment(PPE),as an important guarantee to reduce casualties and maintain military combat effectiveness,is widely developed.This paper systematically reviewed various PPE based on individual combat through literature research and comprehensive discussion,and introduced in detail the latest application progress of PPE in terms of material and technology from three aspects:individual integrated protection system,traditional protection equipment,and intelligent protection equipment,respectively,and discussed in depth the functional improvement and optimization status brought by advanced technology for PPE,focusing on the achievements of individual equipment technology application.Finally,the problems and technical bottlenecks in the development of PPE were analyzed and summarized,and the development trend of PPE were pointed out.The results of the review will provide a forward-looking reference for the current development of individual PPE,and are important guidance for the design and technological innovation of advanced equipment based on the future technological battlefield.

Entry-Level Forward Surgical Team Training Is Associated with Increased Confidence of Primary Combat Surgeons

Background: In recent years, we have established an entry-level Forward Surgical Team (FST) training program in a Chinese military medical university for the 5th grade undergraduates, who would be deployed to different military medical services as primary combat surgeons. This study aimed to assess the role of this pre-service training in improving their confidence with combat medical skills, after several years since they received the training. Methods: We conducted a nationwide survey of 239 primary combat surgeons who have ever participated in an entry-level FST training program before deployment between June 2016 and June 2020, which was for evaluating on a 5-point Likert scale the benefits of entry-level FST training and conventional surgery training in improving their confidence with combat medical skills. The difference in scores was compared using the student t-test. Significance was considered as P Results: The total score was significantly higher for entry-level FST training than that for conventional surgery training (30.76 ± 4.33 vs. 28.95 ± 4.80, P There was no significant difference between the training for surgical skills confidence scores (18.03 ± 8.04 vs. 17.51 ± 8.30, P = 0.098), but for non-technical skills, the score of entry-level FST training was significantly higher than that of conventional surgery training (12.73 ± 5.39 vs. 11.44 ± 5.62, P The distributions of confidence scores were different under various subgroups by demographics. There were no significant differences in scores between the two training in all specific surgical skill sets except “life-saving surgery” (P = 0.011). Scores of all 4 non-technical skill sets were significantly higher for entry-level FST than those for conventional surgery training (P Conclusions: The training should be considered as an essential strategy to improve confidence in combat medical skills, especially life-saving surgery and non-technical skills, for primary combat surgeons.

基于“马赛克战”的水下有人-无人作战使用探讨

通过对“马赛克战”概念的背景和特点进行分析,提出基于马赛克的水下攻防作战运用构想,构建基于马赛克的水下有人-无人作战网络,分析多域背景下水下OODA作战网络的构成,利用OODA作战环理论对水下马赛克作战网络作战能力进行分析,并提出水下马赛克作战网络对抗单个平台和对抗作战网络的使用方法。最后,从通信组网、集群使用、人工智能、察打一体等4个方面对“马赛克战”概念下水下无人航行器装备使用提出建议。

联合全域作战态势下对战伤救治能力建设需求的思考

21世纪初,以人工智能为代表的颠覆性技术的突破与创新深刻影响了人类社会的发展,也成为引领全球新军事变革的核心组成部分。挽救伤员生命与技术创新是军事医学发展的原动力。如何在新的作战样式和态势下成功救治伤病员,是军事医学必须回答的问题。联合全域作战是一种全新的军事理念,代表着未来作战样式的发展方向,对未来战争具有很强的战略指导意义,也对战伤救治提出了更高的要求。“白金10分钟,黄金1小时”是战场一线救治时效的根本法则,战伤救治必须服务于联合全域作战需求与勤务要求。本文就联合全域作战态势下战伤救治能力建设的需求问题进行初步探讨。

基于强化学习的对抗意图识别

未来智能化战争复杂多变,敌我双方往往以对抗博弈情况出现,当我方作为攻击者时,如何有效隐藏我方意图实施有效打击,以及我方作为防御者时如何识别敌方意图实施有效防御,对战争局势的走向起到决定性作用。为把握战争局势,做到“知己知彼,百战不殆”,基于强化学习方法针对对抗意图识别问题提出了解决方案。仿真结果表明,基于强化学习被识别者可以有效地进行意图欺骗,而识别者可以通过意图识别设计方法有效地迫使被识别者暴露意图,从而采取攻防措施。

战教融合视域下军校实战化教学改革探讨

在明确战教融合视域要求的基础上,对军校实战化教学的内涵进行说明。针对当前实战化教学中存在的问题,从思想认识、教学内容、教学保障条件、教学模式和教员队伍五个方面分析与实战化教学不相适应的原因,对应提出深化思想认识、优化教学内容、改善教学保障条件、精选教学模式和提升教员晓战能力的改革建议措施,以期为军校提高实战化教学效果提供有益借鉴。

新形势下航天装备环境试验技术的未来发展趋势

从航天装备面临的实战化考核、数字化建设和低成本控制等新形势与新要求出发,梳理了环境试验技术在航天装备试验鉴定中存在的实战化考核能力不足、全寿命周期环境效应考核不充分等技术难题,结合航天装备环境试验的国内外发展现状,系统分析了航天装备环境试验技术的未来发展趋势,试验环境综合化、试验设计一体化、试验手段数字化在提升航天装备环境试验水平、缩短装备试验周期、降低试验成本方面具有显著优势,也是航天装备环境试验未来发展的重要方向。

美军军事仿真LVC系统发展与启示

针对当前多军兵种联合作战实战化训练的需求,建模和仿真领域的未来目标是创建统一的真实-虚拟-构造(LVC)集成架构,支持快速集成模型和开展仿真,用于联合作战训练、战术协同、拟制作战计划和评估等。通过探究LVC概念内涵和应用领域,分析美军LVC训练网络发展计划,梳理了分布式任务作战、联合仿真环境和美海军模拟训练三个典型LVC系统应用范例。借鉴美军应用LVC解决联合训练的经验做法,给出加快构建我军一体化联合训练的启示,为我军联合训练环境建设提供有价值的参考。

俄罗斯预警机装备发展过程研究及展望

俄罗斯预警机继承于苏联,受技术水平、政治和经济等多重因素影响,其现役装备能力已落后于世界先进水平.首先通过对苏联/俄罗斯研发的主要预警机装备情况进行梳理;然后回顾了俄罗斯预警机装备发展历程,对其装备能力现状、技术特点进行了总结;最后结合俄乌冲突中两架A-50预警机被击落事件,对俄罗斯预警机装备未来发展趋势进行展望.

面向无人机海上侦察任务的自适应粒度分解策略研究

无人机海上侦察作战任务自动规划的前提是任务分解,分解策略的通用性和分解粒度的合理性直接影响任务规划的速度和实际效果。对无人机海上侦察任务具有随机性强、复杂度高、动态性明显等特点,提出了一种基于军事领域知识库的无人机海上侦察任务自适应粒度分解策略,构建了无人机海上侦察领域知识图谱和自适应粒度任务分解模型,并以海上编队护航背景下的无人机侦察任务为例,实施了任务分解策略验证,结果表明分解策略较传统任务分解方法具有更佳的稳定性和适应性。

无人空战指控建模仿真方法研究

针对当前无人空战仿真中,缺乏支持全流程作战指挥控制建模与仿真这一问题,结合当前无人空战仿真系统相关研究,系统梳理和研究指控模型构建方法,通过探索不同仿真层级不同类型的无人机指挥控制建模原理和方法,设计并且构建了一套适合指挥信息系统装备特点的指控模型仿真系统。通过进行符合预设约束的想定设定,对目标作战任务进行了效能评估和对比,可为无人空战相关指挥信息系统装备的论证、评估方式提供理论和现实指导。

飞行学员体能组训模式改革与探究

基于战场和作战任务对军人综合能力的需求,尤其是实战化军事训练对飞行学员能力素质提出的新要求,使得当前组训模式将体能、技能、智能训练置于战术背景下,整体设计、综合实施,加大应用训练、协同训练力度。结合陆军飞行学员岗位任职要求,针对飞行学员的特点及体能组训模式生成需求,遵循战训融合式发展理念,优化课程教学训练内容,提出训练中采用“课目群”训练方法,有效提升学员体能素质综合能力和训练效果,全面有效推进教学训练向实战化聚焦,进而提升陆军兵种航空体育训练对战斗力的贡献度。

空战目标轨迹预测技术研究综述

信息化空战对抗中,快速获取和有效利用对手信息的一方,能够准确地预测对手的运动轨迹,更加迅速地完成OODA循环,进而取得空战优势。本文研究空战目标轨迹预测技术,以时序变化特征为研究视角,分别对基于物理、机器学习和深度学习的轨迹预测技术开展分析和总结,进一步归纳物理因素、空战因素和交互因素等作为轨迹预测模型的输入,单模态轨迹、多模态轨迹和行为意图三类作为模型的输出,并对未来智能化空战中的目标轨迹预测技术进行展望。

基于改进蚁群算法和共用航段的搜索救援通道规划

针对战斗搜救任务场景中,空中搜索救援等待区至跳伞区之间的往返路径规划问题,提出一种基于改进蚁群算法和共用航段的搜索救援通道规划方法。首先,针对基本蚁群算法易出现局部极值及收敛速度慢等缺点,对启发函数、状态转移策略、信息素更新策略进行了改进。然后,针对单起点多目标点的路径规划问题,提出了设置分航点以生成共用航段的搜索救援通道规划方法,并以总路径长度表征任务准备负荷量与分别规划航路进行对比。实验结果表明,在本文测试环境下,改进蚁群算法规划的路径长度分别缩减15.4%和14.2%,搜索救援通道的任务准备负荷量减少25.4%。所提出的搜索救援通道规划方法有一定的理论和应用价值。