Mastering air combat game with deep reinforcement learning

Reinforcement learning has been applied to air combat problems in recent years,and the idea of curriculum learning is often used for reinforcement learning,but traditional curriculum learning suffers from the problem of plasticity loss in neural networks.Plasticity loss is the difficulty of learning new knowledge after the network has converged.To this end,we propose a motivational curriculum learning distributed proximal policy optimization(MCLDPPO)algorithm,through which trained agents can significantly outperform the predictive game tree and mainstream reinforcement learning methods.The motivational curriculum learning is designed to help the agent gradually improve its combat ability by observing the agent's unsatisfactory performance and providing appropriate rewards as a guide.Furthermore,a complete tactical maneuver is encapsulated based on the existing air combat knowledge,and through the flexible use of these maneuvers,some tactics beyond human knowledge can be realized.In addition,we designed an interruption mechanism for the agent to increase the frequency of decisionmaking when the agent faces an emergency.When the number of threats received by the agent changes,the current action is interrupted in order to reacquire observations and make decisions again.Using the interruption mechanism can significantly improve the performance of the agent.To simulate actual air combat better,we use digital twin technology to simulate real air battles and propose a parallel battlefield mechanism that can run multiple simulation environments simultaneously,effectively improving data throughput.The experimental results demonstrate that the agent can fully utilize the situational information to make reasonable decisions and provide tactical adaptation in the air combat,verifying the effectiveness of the algorithmic framework proposed in this paper.

Optimal confrontation position selecting games model and its application to one-on-one air combat

In the air combat process,confrontation position is the critical factor to determine the confrontation situation,attack effect and escape probability of UAVs.Therefore,selecting the optimal confrontation position becomes the primary goal of maneuver decision-making.By taking the position as the UAV’s maneuver strategy,this paper constructs the optimal confrontation position selecting games(OCPSGs)model.In the OCPSGs model,the payoff function of each UAV is defined by the difference between the comprehensive advantages of both sides,and the strategy space of each UAV at every step is defined by its accessible space determined by the maneuverability.Then we design the limit approximation of mixed strategy Nash equilibrium(LAMSNQ)algorithm,which provides a method to determine the optimal probability distribution of positions in the strategy space.In the simulation phase,we assume the motions on three directions are independent and the strategy space is a cuboid to simplify the model.Several simulations are performed to verify the feasibility,effectiveness and stability of the algorithm.

Combating Cholera

Lying in her makeshift hospital bed,Joyce Tembo thanked medical personnel for evacuating her to the designated national cholera treatment centre,6 km north of Zambia’s capital Lusaka.She was recently diagnosed with diarrhoeal disease.Tembo,43,commended the medical sta!stationed at the treatment centre for their great service to thousands of patients,especially women and children seeking urgent treatment.“I am very grateful to the Chinese doctors who attended to me as soon as the ambulance rushed me to the clinic where I received urgent treatment;they have really saved my life,”Tembo told ChinAfrica.But not all residents in her community are as lucky as her.Many in the densely populated slums die every day due to the area’s poor sanitation-one of the major causes of the cholera outbreak.

Cooperative maneuver decision making for multi-UAV air combat based on incomplete information dynamic game

Cooperative autonomous air combat of multiple unmanned aerial vehicles(UAVs)is one of the main combat modes in future air warfare,which becomes even more complicated with highly changeable situation and uncertain information of the opponents.As such,this paper presents a cooperative decision-making method based on incomplete information dynamic game to generate maneuver strategies for multiple UAVs in air combat.Firstly,a cooperative situation assessment model is presented to measure the overall combat situation.Secondly,an incomplete information dynamic game model is proposed to model the dynamic process of air combat,and a dynamic Bayesian network is designed to infer the tactical intention of the opponent.Then a reinforcement learning framework based on multiagent deep deterministic policy gradient is established to obtain the perfect Bayes-Nash equilibrium solution of the air combat game model.Finally,a series of simulations are conducted to verify the effectiveness of the proposed method,and the simulation results show effective synergies and cooperative tactics.

The personal protective equipment(PPE)based on individual combat:A systematic review and trend analysis

With the development of ordnance technology,the survival and safety of individual combatants in hightech warfare are under serious threat,and the Personal Protective Equipment(PPE),as an important guarantee to reduce casualties and maintain military combat effectiveness,is widely developed.This paper systematically reviewed various PPE based on individual combat through literature research and comprehensive discussion,and introduced in detail the latest application progress of PPE in terms of material and technology from three aspects:individual integrated protection system,traditional protection equipment,and intelligent protection equipment,respectively,and discussed in depth the functional improvement and optimization status brought by advanced technology for PPE,focusing on the achievements of individual equipment technology application.Finally,the problems and technical bottlenecks in the development of PPE were analyzed and summarized,and the development trend of PPE were pointed out.The results of the review will provide a forward-looking reference for the current development of individual PPE,and are important guidance for the design and technological innovation of advanced equipment based on the future technological battlefield.

Autonomous air combat decision-making of UAV based on parallel self-play reinforcement learning

Aiming at addressing the problem of manoeuvring decision-making in UAV air combat,this study establishes a one-to-one air combat model,defines missile attack areas,and uses the non-deterministic policy Soft-Actor-Critic(SAC)algorithm in deep reinforcement learning to construct a decision model to realize the manoeuvring process.At the same time,the complexity of the proposed algorithm is calculated,and the stability of the closed-loop system of air combat decision-making controlled by neural network is analysed by the Lyapunov function.This study defines the UAV air combat process as a gaming process and proposes a Parallel Self-Play training SAC algorithm(PSP-SAC)to improve the generalisation performance of UAV control decisions.Simulation results have shown that the proposed algorithm can realize sample sharing and policy sharing in multiple combat environments and can significantly improve the generalisation ability of the model compared to independent training.

Entry-Level Forward Surgical Team Training Is Associated with Increased Confidence of Primary Combat Surgeons

Background: In recent years, we have established an entry-level Forward Surgical Team (FST) training program in a Chinese military medical university for the 5th grade undergraduates, who would be deployed to different military medical services as primary combat surgeons. This study aimed to assess the role of this pre-service training in improving their confidence with combat medical skills, after several years since they received the training. Methods: We conducted a nationwide survey of 239 primary combat surgeons who have ever participated in an entry-level FST training program before deployment between June 2016 and June 2020, which was for evaluating on a 5-point Likert scale the benefits of entry-level FST training and conventional surgery training in improving their confidence with combat medical skills. The difference in scores was compared using the student t-test. Significance was considered as P Results: The total score was significantly higher for entry-level FST training than that for conventional surgery training (30.76 ± 4.33 vs. 28.95 ± 4.80, P There was no significant difference between the training for surgical skills confidence scores (18.03 ± 8.04 vs. 17.51 ± 8.30, P = 0.098), but for non-technical skills, the score of entry-level FST training was significantly higher than that of conventional surgery training (12.73 ± 5.39 vs. 11.44 ± 5.62, P The distributions of confidence scores were different under various subgroups by demographics. There were no significant differences in scores between the two training in all specific surgical skill sets except “life-saving surgery” (P = 0.011). Scores of all 4 non-technical skill sets were significantly higher for entry-level FST than those for conventional surgery training (P Conclusions: The training should be considered as an essential strategy to improve confidence in combat medical skills, especially life-saving surgery and non-technical skills, for primary combat surgeons.

基于“马赛克战”的水下有人-无人作战使用探讨

通过对“马赛克战”概念的背景和特点进行分析,提出基于马赛克的水下攻防作战运用构想,构建基于马赛克的水下有人-无人作战网络,分析多域背景下水下OODA作战网络的构成,利用OODA作战环理论对水下马赛克作战网络作战能力进行分析,并提出水下马赛克作战网络对抗单个平台和对抗作战网络的使用方法。最后,从通信组网、集群使用、人工智能、察打一体等4个方面对“马赛克战”概念下水下无人航行器装备使用提出建议。

联合全域作战态势下对战伤救治能力建设需求的思考

21世纪初,以人工智能为代表的颠覆性技术的突破与创新深刻影响了人类社会的发展,也成为引领全球新军事变革的核心组成部分。挽救伤员生命与技术创新是军事医学发展的原动力。如何在新的作战样式和态势下成功救治伤病员,是军事医学必须回答的问题。联合全域作战是一种全新的军事理念,代表着未来作战样式的发展方向,对未来战争具有很强的战略指导意义,也对战伤救治提出了更高的要求。“白金10分钟,黄金1小时”是战场一线救治时效的根本法则,战伤救治必须服务于联合全域作战需求与勤务要求。本文就联合全域作战态势下战伤救治能力建设的需求问题进行初步探讨。

新形势下航天装备环境试验技术的未来发展趋势

从航天装备面临的实战化考核、数字化建设和低成本控制等新形势与新要求出发,梳理了环境试验技术在航天装备试验鉴定中存在的实战化考核能力不足、全寿命周期环境效应考核不充分等技术难题,结合航天装备环境试验的国内外发展现状,系统分析了航天装备环境试验技术的未来发展趋势,试验环境综合化、试验设计一体化、试验手段数字化在提升航天装备环境试验水平、缩短装备试验周期、降低试验成本方面具有显著优势,也是航天装备环境试验未来发展的重要方向。

基于改进蚁群算法和共用航段的搜索救援通道规划

针对战斗搜救任务场景中,空中搜索救援等待区至跳伞区之间的往返路径规划问题,提出一种基于改进蚁群算法和共用航段的搜索救援通道规划方法。首先,针对基本蚁群算法易出现局部极值及收敛速度慢等缺点,对启发函数、状态转移策略、信息素更新策略进行了改进。然后,针对单起点多目标点的路径规划问题,提出了设置分航点以生成共用航段的搜索救援通道规划方法,并以总路径长度表征任务准备负荷量与分别规划航路进行对比。实验结果表明,在本文测试环境下,改进蚁群算法规划的路径长度分别缩减15.4%和14.2%,搜索救援通道的任务准备负荷量减少25.4%。所提出的搜索救援通道规划方法有一定的理论和应用价值。

美军军事仿真LVC系统发展与启示

针对当前多军兵种联合作战实战化训练的需求,建模和仿真领域的未来目标是创建统一的真实-虚拟-构造(LVC)集成架构,支持快速集成模型和开展仿真,用于联合作战训练、战术协同、拟制作战计划和评估等。通过探究LVC概念内涵和应用领域,分析美军LVC训练网络发展计划,梳理了分布式任务作战、联合仿真环境和美海军模拟训练三个典型LVC系统应用范例。借鉴美军应用LVC解决联合训练的经验做法,给出加快构建我军一体化联合训练的启示,为我军联合训练环境建设提供有价值的参考。

面向无人机海上侦察任务的自适应粒度分解策略研究

无人机海上侦察作战任务自动规划的前提是任务分解,分解策略的通用性和分解粒度的合理性直接影响任务规划的速度和实际效果。对无人机海上侦察任务具有随机性强、复杂度高、动态性明显等特点,提出了一种基于军事领域知识库的无人机海上侦察任务自适应粒度分解策略,构建了无人机海上侦察领域知识图谱和自适应粒度任务分解模型,并以海上编队护航背景下的无人机侦察任务为例,实施了任务分解策略验证,结果表明分解策略较传统任务分解方法具有更佳的稳定性和适应性。

基于强化学习的对抗意图识别

未来智能化战争复杂多变,敌我双方往往以对抗博弈情况出现,当我方作为攻击者时,如何有效隐藏我方意图实施有效打击,以及我方作为防御者时如何识别敌方意图实施有效防御,对战争局势的走向起到决定性作用。为把握战争局势,做到“知己知彼,百战不殆”,基于强化学习方法针对对抗意图识别问题提出了解决方案。仿真结果表明,基于强化学习被识别者可以有效地进行意图欺骗,而识别者可以通过意图识别设计方法有效地迫使被识别者暴露意图,从而采取攻防措施。

基于分组教与学的无人战斗机自适应路径规划

针对无人战斗机(unmanned combat air vehicle,UCAV)处于存在威胁区域的战场中路径规划问题,提出一种基于分组教与学算法的UCAV自适应路径规划方法。通过分析UCAV路径评价指标,提出一种自适应的UCAV路径评价模型,根据作战环境规划出距离短、威胁小的任务路径。针对教与学算法寻优精度低、耗时长的问题,提出一种分组教与学算法,引入动态分组和高斯分布扰动策略,提高算法寻优性能。通过仿真实验,该方案求解的最优路径更短且安全。

无人空战指控建模仿真方法研究

针对当前无人空战仿真中,缺乏支持全流程作战指挥控制建模与仿真这一问题,结合当前无人空战仿真系统相关研究,系统梳理和研究指控模型构建方法,通过探索不同仿真层级不同类型的无人机指挥控制建模原理和方法,设计并且构建了一套适合指挥信息系统装备特点的指控模型仿真系统。通过进行符合预设约束的想定设定,对目标作战任务进行了效能评估和对比,可为无人空战相关指挥信息系统装备的论证、评估方式提供理论和现实指导。

基于Multi-Agent的无人机集群体系自主作战系统设计

针对无人集群自主作战体系设计中的关键问题,提出基于Multi-Agent的无人集群自主作战系统设计方法。建立无人集群各节点的Agent模型及其推演规则;对于仿真系统模块化和通用化的需求,设计系统互操作式接口和无人集群自主作战的交互关系;开展无人集群系统仿真推演验证。仿真结果表明,所提设计方案不仅能够有效开展并完成自主作战网络生成-集群演化-效能评估的全过程动态演示验证,而且能够通过重复随机试验进一步评估无人集群的协同作战效能,最后总结了集群协同作战的策略和经验。

无人机自主作战效能评估技术研究综述

无人机自主作战效能评估是衡量无人机在特定作战条件下能否运用自主能力完成预定目标任务的有效途径,是无人机自主作战技术论证和装备体系建设发展的核心问题。从无人机作战效能基本问题出发,首先,剖析无人机自主性与自主能力以及自主作战能力与自主作战效能的基本概念,给出无人机自主作战效能评估的基本流程;其次,分别从任务视角和原理角度对无人装备作战效能评估指标体系、评估方法的研究现状进行综述,归纳总结3类指标体系构建的现状和4类效能评估方法进展;最后,梳理当前评估中存在的问题,展望无人机效能评估技术的发展趋势。相关研究可为后续无人机自主作战效能评估提供技术参考和方向指导,具有重要的军事意义。

卫星任务管控系统实战化检验分析与研究

随着航天系统运用由业务运行模式向作战使用模式转变,对卫星任务管控系统的产品质量提出了更高的要求。现行的检验试验基于满足产品的技术性能指标,检验条件单一、检验流程固定。为有效预防系统在使用中出现“不好用”“不实用”的现象,深入分析系统的实战化要求,构建了测试过程模型,提出了在上线前对系统进行全链路压测的测试方法。通过多角度想定系统应用场景,选取多样本数据,实现了最大限度地模拟实战环境,检验了系统的作战使用效能,为信息系统的实战化检验提供了一种有效的检验模式和方法。

国际人道法中“直接参加敌对行动”的功能区分标准及其法理反思

无论是在历史上还是当今现实中,大量的平民在事实上都可能会以不同角色卷入到具体的武装冲突之中。随着现代武装冲突在思维范式、时空语境、科技条件等方面呈现出一系列新型的特点,“平民直接参加敌对行动”这一现象已经更加广泛地出现。与此同时,作为国际人道法基础的区分原则也正面临着越来越严重的法理挑战。传统的静态化的身份区分标准在解释力方面已经陷入困境。相比而言,建构一种以“持续作战职责”作为基准的功能区分理论就显得尤为必要。以“功能区分标准”为基础廓清和厘定“直接参加敌对行动”的法理概念有利于划分平民与直接参加敌对行动人员的界限。为了更好地落实区分原则在“直接参加敌对行动”中的应用,在红十字国际委员会《国际人道法中直接参加敌对行动定义的解释性指南》的基础之上,现代国际人道法仍需强化功能区分标准的法理价值、澄清“持续作战职责”的法理要素、划定直接与间接参加敌对行动的功能边界,从而使得国际人道法的解释体系在人道主义原则与军事必要性之间寻求一个可操作化的精巧平衡。