Mastering air combat game with deep reinforcement learning

Reinforcement learning has been applied to air combat problems in recent years,and the idea of curriculum learning is often used for reinforcement learning,but traditional curriculum learning suffers from the problem of plasticity loss in neural networks.Plasticity loss is the difficulty of learning new knowledge after the network has converged.To this end,we propose a motivational curriculum learning distributed proximal policy optimization(MCLDPPO)algorithm,through which trained agents can significantly outperform the predictive game tree and mainstream reinforcement learning methods.The motivational curriculum learning is designed to help the agent gradually improve its combat ability by observing the agent's unsatisfactory performance and providing appropriate rewards as a guide.Furthermore,a complete tactical maneuver is encapsulated based on the existing air combat knowledge,and through the flexible use of these maneuvers,some tactics beyond human knowledge can be realized.In addition,we designed an interruption mechanism for the agent to increase the frequency of decisionmaking when the agent faces an emergency.When the number of threats received by the agent changes,the current action is interrupted in order to reacquire observations and make decisions again.Using the interruption mechanism can significantly improve the performance of the agent.To simulate actual air combat better,we use digital twin technology to simulate real air battles and propose a parallel battlefield mechanism that can run multiple simulation environments simultaneously,effectively improving data throughput.The experimental results demonstrate that the agent can fully utilize the situational information to make reasonable decisions and provide tactical adaptation in the air combat,verifying the effectiveness of the algorithmic framework proposed in this paper.

Optimal confrontation position selecting games model and its application to one-on-one air combat

In the air combat process,confrontation position is the critical factor to determine the confrontation situation,attack effect and escape probability of UAVs.Therefore,selecting the optimal confrontation position becomes the primary goal of maneuver decision-making.By taking the position as the UAV’s maneuver strategy,this paper constructs the optimal confrontation position selecting games(OCPSGs)model.In the OCPSGs model,the payoff function of each UAV is defined by the difference between the comprehensive advantages of both sides,and the strategy space of each UAV at every step is defined by its accessible space determined by the maneuverability.Then we design the limit approximation of mixed strategy Nash equilibrium(LAMSNQ)algorithm,which provides a method to determine the optimal probability distribution of positions in the strategy space.In the simulation phase,we assume the motions on three directions are independent and the strategy space is a cuboid to simplify the model.Several simulations are performed to verify the feasibility,effectiveness and stability of the algorithm.

The effectiveness of the department of defense's field manual 3-11 in detecting,deterring and degrading the breach of a combat base by a human-borne with bioagent(HBBA):perceptions of security personnel

Background: The department of defense's field manual(FM) 3-11 is among the military's field manuals for preparing for, reacting to and recovering from chemical, biological, radiological and nuclear attacks. Since post 9-11, U.S. military service members have been deployed in the global war on terrorism. This study attempted to determine the effectiveness of the FM 3-11 in detecting, deterring or preventing a human-borne with bioagent(HBBA) terrorist breach at an entry control point(ECP).Methods: This time-specific, cross-sectional study disseminated a validated survey tool with Cronbach's α>0.82 to respondents who have had antiterrorism training and combat ECP experience. The return rate was greater than 75.0%; however, many of the respondents failed to meet the inclusion criteria. Consequently, only 26 questionnaires were included in the sample.Results: The results revealed that while over 60.0% of the respondents either strongly agreed or agreed that biointelligence, the deployment of biodetectors and the use of biowarning systems could be effective in preventing an ECP breach by a terrorist with a bioagent, the use of protective equipment and immunization to decontaminate service members or other tactics, techniques and procedures(TTPs) would never prevent a breach. A large percentage of respondents claimed that soldiers at the ECP lacked the devices or the knowledge to detect an HBBA at an ECP, and 72.0% suggested modifying current ECP TTPs to include education, training and equipment for security personnel at military base ECPs.Conclusion: If obtained from appropriate sources and communicated to the personnel at the ECP in an effective or timely manner, the possible effectiveness of certain TTPs in the FM 3-11, specifically FM 3-11.86(intelligence), might increase.

A function-based behavioral modeling method for air combat simulation

Today’s air combat has reached a high level of uncertainty where continuous or discrete variables with crisp values cannot be properly represented using fuzzy sets. With a set of membership functions, fuzzy logic is well-suited to tackle such complex states and actions. However, it is not necessary to fuzzify the variables that have definite discrete semantics.Hence, the aim of this study is to improve the level of model abstraction by proposing multiple levels of cascaded hierarchical structures from the perspective of function, namely, the functional decision tree. This method is developed to represent behavioral modeling of air combat systems, and its metamodel,execution mechanism, and code generation can provide a sound basis for function-based behavioral modeling. As a proof of concept, an air combat simulation is developed to validate this method and the results show that the fighter Alpha built using the proposed framework provides better performance than that using default scripts.

Combating Cholera

Lying in her makeshift hospital bed,Joyce Tembo thanked medical personnel for evacuating her to the designated national cholera treatment centre,6 km north of Zambia’s capital Lusaka.She was recently diagnosed with diarrhoeal disease.Tembo,43,commended the medical sta!stationed at the treatment centre for their great service to thousands of patients,especially women and children seeking urgent treatment.“I am very grateful to the Chinese doctors who attended to me as soon as the ambulance rushed me to the clinic where I received urgent treatment;they have really saved my life,”Tembo told ChinAfrica.But not all residents in her community are as lucky as her.Many in the densely populated slums die every day due to the area’s poor sanitation-one of the major causes of the cholera outbreak.

The personal protective equipment(PPE)based on individual combat:A systematic review and trend analysis

With the development of ordnance technology,the survival and safety of individual combatants in hightech warfare are under serious threat,and the Personal Protective Equipment(PPE),as an important guarantee to reduce casualties and maintain military combat effectiveness,is widely developed.This paper systematically reviewed various PPE based on individual combat through literature research and comprehensive discussion,and introduced in detail the latest application progress of PPE in terms of material and technology from three aspects:individual integrated protection system,traditional protection equipment,and intelligent protection equipment,respectively,and discussed in depth the functional improvement and optimization status brought by advanced technology for PPE,focusing on the achievements of individual equipment technology application.Finally,the problems and technical bottlenecks in the development of PPE were analyzed and summarized,and the development trend of PPE were pointed out.The results of the review will provide a forward-looking reference for the current development of individual PPE,and are important guidance for the design and technological innovation of advanced equipment based on the future technological battlefield.

Peritraumatic Behavior Questionnaire- Observer Rated: Validation of the objective version of a measure for combatrelated peritraumatic stress

AIM: To validate the first third-person-rated measure assessing combat-related peritraumatic stress symptoms and evaluate its psychometric properties and war-zone applicability.METHODS: The valid assessment of peritraumatic symptoms in the theater of military operations represents a significant challenge in combat-related, mental health research, which mainly relies on retrospective, subjective self-report ratings. This longitudinal observational study used data from actively deployed troops to correlate third-person observer ratings of deployment peritraumatic behaviors [Peritraumatic Behavior Questionnaire- Observer Rated(PBQ-OR)] collected on a bimonthly basis with post-deployment(1-wk follow-up) ratings of the previously validated PBQ self-rate version(PBQ-SR), and(3-mo follow-up) clinician assessed and self-report posttraumatic stress disorder(PTSD) symptoms(Clinician Administered PTSD Scale, PTSD Checklist). Cronbach's alpha(α) and correlation coefficients were calculated to assess internal reliability and concurrent validity respectively. RESULTS: Eight hundred and sixty male Marines were included in this study after signing informed consents at pre-deployment(mean age 23.2 ± 2.6 years). Although our findings were limited by an overall sparse return rate of PBQ-OR ratings, the main results indicate satisfactory psychometric properties with good internal consistency for the PBQ-OR(α = 0.88) and high convergent and concurrent validity with 1-wk post-deployment PBQSR ratings and 3-mo posttraumatic stress symptoms. Overall, later PBQ-OR report date was associated with higher correlation between PBQ-OR and postdeployment measures. Kappa analysis between PBQOR and PBQ-SR single items, showed best agreement in questions relating of mortal peril, desire for revenge, and experience of intense physical reactions. Logistic regression demonstrated satisfactory predictive validity of PBQ-OR total score with respect to PTSD caseness(OR = 1.0513; 95%CI: 1.011-1.093; P = 0.02).CONCLUSION: Since no comparable tools have been developed, PBQ-OR could be valuable as real-time screening tool for earlier detection of Service Members at risk.

Cooperative maneuver decision making for multi-UAV air combat based on incomplete information dynamic game

Cooperative autonomous air combat of multiple unmanned aerial vehicles(UAVs)is one of the main combat modes in future air warfare,which becomes even more complicated with highly changeable situation and uncertain information of the opponents.As such,this paper presents a cooperative decision-making method based on incomplete information dynamic game to generate maneuver strategies for multiple UAVs in air combat.Firstly,a cooperative situation assessment model is presented to measure the overall combat situation.Secondly,an incomplete information dynamic game model is proposed to model the dynamic process of air combat,and a dynamic Bayesian network is designed to infer the tactical intention of the opponent.Then a reinforcement learning framework based on multiagent deep deterministic policy gradient is established to obtain the perfect Bayes-Nash equilibrium solution of the air combat game model.Finally,a series of simulations are conducted to verify the effectiveness of the proposed method,and the simulation results show effective synergies and cooperative tactics.

Autonomous air combat decision-making of UAV based on parallel self-play reinforcement learning

Aiming at addressing the problem of manoeuvring decision-making in UAV air combat,this study establishes a one-to-one air combat model,defines missile attack areas,and uses the non-deterministic policy Soft-Actor-Critic(SAC)algorithm in deep reinforcement learning to construct a decision model to realize the manoeuvring process.At the same time,the complexity of the proposed algorithm is calculated,and the stability of the closed-loop system of air combat decision-making controlled by neural network is analysed by the Lyapunov function.This study defines the UAV air combat process as a gaming process and proposes a Parallel Self-Play training SAC algorithm(PSP-SAC)to improve the generalisation performance of UAV control decisions.Simulation results have shown that the proposed algorithm can realize sample sharing and policy sharing in multiple combat environments and can significantly improve the generalisation ability of the model compared to independent training.

Entry-Level Forward Surgical Team Training Is Associated with Increased Confidence of Primary Combat Surgeons

Background: In recent years, we have established an entry-level Forward Surgical Team (FST) training program in a Chinese military medical university for the 5th grade undergraduates, who would be deployed to different military medical services as primary combat surgeons. This study aimed to assess the role of this pre-service training in improving their confidence with combat medical skills, after several years since they received the training. Methods: We conducted a nationwide survey of 239 primary combat surgeons who have ever participated in an entry-level FST training program before deployment between June 2016 and June 2020, which was for evaluating on a 5-point Likert scale the benefits of entry-level FST training and conventional surgery training in improving their confidence with combat medical skills. The difference in scores was compared using the student t-test. Significance was considered as P Results: The total score was significantly higher for entry-level FST training than that for conventional surgery training (30.76 ± 4.33 vs. 28.95 ± 4.80, P There was no significant difference between the training for surgical skills confidence scores (18.03 ± 8.04 vs. 17.51 ± 8.30, P = 0.098), but for non-technical skills, the score of entry-level FST training was significantly higher than that of conventional surgery training (12.73 ± 5.39 vs. 11.44 ± 5.62, P The distributions of confidence scores were different under various subgroups by demographics. There were no significant differences in scores between the two training in all specific surgical skill sets except “life-saving surgery” (P = 0.011). Scores of all 4 non-technical skill sets were significantly higher for entry-level FST than those for conventional surgery training (P Conclusions: The training should be considered as an essential strategy to improve confidence in combat medical skills, especially life-saving surgery and non-technical skills, for primary combat surgeons.

Modeling of Naval Vessels Combat System based on Complex Network

Naval Vessels Combat System is a kind of complex system.The modeling of combat system has become hot issues in the past years.This paper proposed a new method to establish models of combat system based on the theory of Complex Network.The method of modeling considered the operational entities as nodes.It considered flow of information,substance and energy as edges in a network.The research also carries on a simulation to prove the applicability.Ultimately,the paper concluded that this method is applicable and accurate.

Chinese expert consensus on the treatment of modern combat-related spinal injuries

The battlefield treatments of spinal and spinal cord injury vary from civilian settings. However, there is no unified battlefield treatment guidelines for spine trauma in PLA. An expert consensus is reached, based on spine trauma epidemiology and the concepts of battlefield treatment combined with the existing levels of military medical care in modern warfare. Since the specialized treatment for spine trauma are no significant difference between civilian settings and modern war, the first aid, emergency treatment and early treatment of spine trauma are introduced separately in three levels in this consensus. In Level I facilities, the fast and accurate evaluation of spine trauma followed by fixation and stabilization are recommended during the first-aid stage. Re-evaluation, further treatment for possible hemorrhagic shock, dyspnea and infection are recommended at Level II facilities. At Level III facilities, it is recommended to strengthen the intensive care and the prevention of urinary system and lung infection for the wounded with severe spinal injury, however, spinal surgery is not recommended in a battlefield hospital. The grading standard for evidence evaluation and recommendation was used to reach this expert consensus.

Medical rescue of naval combat: challenge and future

There has been no large-scale naval combat in the last 30 years. With the rapid development of battleships, weapons manufacturing and electronic technology, naval combat will present some new characteristics. Additionally, naval combat is facing unprecedented challenges. In this paper, we discuss the topic of medical rescue at sea: what challenges we face and what we could do. The contents discussed in this paper contain battlefield self-aid buddy care, clinical skills, organized health services, medical training and future medical research programs. We also discuss the characteristics of modern naval combat, medical rescue challenges, medical treatment highlights and future developments of medical rescue at sea.

Impact of prehospital medical evacuation (MEDEVAC) transport time on combat mortality in patients with noncompressible torso injury and traumatic amputations: a retrospective study

Background: In combat operations, patients with traumatic injuries require expeditious evacuation to improve survival. Studies have shown that long transport times are associated with increased morbidity and mortality. Limited data exist on the influence of transport time on patient outcomes with specific injury types. The objective of this study was to determine the impact of the duration of time from the initial request for medical evacuation to arrival at a medical treatment facility on morbidity and mortality in casualties with traumatic extremity amputation and noncompressible torso injury(NCTI).Methods: We completed a retrospective review of MEDEVAC patient care records for United States military personnel who sustained traumatic amputations and NCTI during Operation Enduring Freedom between January 2011 and March 2014. We grouped patients as traumatic amputation and NCTI(AMP+NCTI), traumatic amputation only(AMP),and neither AMP nor NCTI(Non-AMP/NCTI). Analysis was performed using chi-squared tests, Fisher's exact tests,Cochran-Armitage Trend tests, Shapiro-Wilks tests, Wilcoxon and Kruskal-Wallis techniques and Cox proportional hazards regression modeling.Results: We reviewed 1267 records, of which 669 had an injury severity score(ISS) of 10 or greater and were included in the analysis. In the study population, 15.5% sustained only amputation injuries(n=104, AMP only), 10.8% sustained amputation and NCTI(n=72, AMP+NCTI), and 73.7% did not sustain either an amputation or an NCTI(n=493,Non-AMP/NCTI). AMP+NCTI had the highest mortality(16.7%) with transport time greater than 60 min. While the AMP+NCTI group had decreasing survival with longer transport times, AMP and Non-AMP/NCTI did not exhibit the same trend.Conclusions: A decreased transport time from the point of injury to a medical treatment facility was associated with decreased mortality in patients who suffered a combination of amputation injury and NCTI. No significant association between transport time and outcomes was found in patients who did not sustain NCTI. Priority for rapid evacuation of combat casualties should be given to those with NCTI.

Analysis on MAV/UAV cooperative combat based on complex network

A formation model of manned/unmanned aerial vehicle(MAV/UAV) collaborative combat can qualitatively and quantitatively analyze the synergistic effects.However,there is currently no effective and appropriate model construction method or theory,and research in the field of collaborative capability evaluation is basically nonexistent.According to the actual conditions of cooperative operations,a new MAV/UAV collaborative combat network model construction method based on a complex network is presented.By analyzing the characteristic parameters of the abstract network,the index system and complex network are combined.Then,a method for evaluating the synergistic effect of the cooperative combat network is developed.This method provides assistance for the verification and evaluation of MAV/UAV collaborative combat.

Formation and adjustment of manned/unmanned combat aerial vehicle cooperative engagement system

Manned combat aerial vehicles （MCAVs）, and un-manned combat aerial vehicles （UCAVs） together form a cooper-ative engagement system to carry out operational mission, whichwill be a new air engagement style in the near future. On the basisof analyzing the structure of the MCAV/UCAV cooperative engage-ment system, this paper divides the unique system into three hi-erarchical levels, respectively, i.e., mission level, task-cluster leveland task level. To solve the formation and adjustment problem ofthe latter two levels, three corresponding mathematical modelsare established. To solve these models, three algorithms calledquantum artificial bee colony （QABC） algorithm, greedy strategy（GS） and two-stage greedy strategy （TSGS） are proposed. Finally,a series of simulation experiments are designed to verify the effec-tiveness and superiority of the proposed algorithms.

A Space Unrestricted Multi-Robot Combat Internet of Things System

Multi-robot cooperation problem has received increasing attention in the research community and has been extensively studied from different aspects. Space constrain problem is a major issue for building a multi-robot system. This con- strain is a major hindrance for the efficient cooperation among robots in multi-robot applications. In this paper, we demonstrate a novel architecture of a multi-robot system without space restriction. Our architecture is based on the Internet of Things technology. We validated the proposed architecture using a case study considering a multi-robot combat application.

A Predator-prey Particle Swarm Optimization Approach to Multiple UCAV Air Combat Modeled by Dynamic Game Theory

Dynamic game theory has received considerable attention as a promising technique for formulating control actions for agents in an extended complex enterprise that involves an adversary. At each decision making step, each side seeks the best scheme with the purpose of maximizing its own objective function. In this paper, a game theoretic approach based on predatorprey particle swarm optimization (PP-PSO) is presented, and the dynamic task assignment problem for multiple unmanned combat aerial vehicles (UCAVs) in military operation is decomposed and modeled as a two-player game at each decision stage. The optimal assignment scheme of each stage is regarded as a mixed Nash equilibrium, which can be solved by using the PP-PSO. The effectiveness of our proposed methodology is verified by a typical example of an air military operation that involves two opposing forces: the attacking force Red and the defense force Blue. © 2014 Chinese Association of Automation.

A Multi-UCAV cooperative occupation method based on weapon engagement zones for beyond-visual-range air combat

Recent advances in on-board radar and missile capabilities,combined with individual payload limitations,have led to increased interest in the use of unmanned combat aerial vehicles(UCAVs)for cooperative occupation during beyond-visual-range(BVR)air combat.However,prior research on occupational decision-making in BVR air combat has mostly been limited to one-on-one scenarios.As such,this study presents a practical cooperative occupation decision-making methodology for use with multiple UCAVs.The weapon engagement zone(WEZ)and combat geometry were first used to develop an advantage function for situational assessment of one-on-one engagement.An encircling advantage function was then designed to represent the cooperation of UCAVs,thereby establishing a cooperative occupation model.The corresponding objective function was derived from the one-on-one engagement advantage function and the encircling advantage function.The resulting model exhibited similarities to a mixed-integer nonlinear programming(MINLP)problem.As such,an improved discrete particle swarm optimization(DPSO)algorithm was used to identify a solution.The occupation process was then converted into a formation switching task as part of the cooperative occupation model.A series of simulations were conducted to verify occupational solutions in varying situations,including two-on-two engagement.Simulated results showed these solutions varied with initial conditions and weighting coefficients.This occupation process,based on formation switching,effectively demonstrates the viability of the proposed technique.These cooperative occupation results could provide a theoretical framework for subsequent research in cooperative BVR air combat.

Autonomous maneuver decision-making for a UCAV in short-range aerial combat based on an MS-DDQN algorithm

To solve the problem of realizing autonomous aerial combat decision-making for unmanned combat aerial vehicles(UCAVs) rapidly and accurately in an uncertain environment, this paper proposes a decision-making method based on an improved deep reinforcement learning(DRL) algorithm: the multistep double deep Q-network(MS-DDQN) algorithm. First, a six-degree-of-freedom UCAV model based on an aircraft control system is established on a simulation platform, and the situation assessment functions of the UCAV and its target are established by considering their angles, altitudes, environments, missile attack performances, and UCAV performance. By controlling the flight path angle, roll angle, and flight velocity, 27 common basic actions are designed. On this basis, aiming to overcome the defects of traditional DRL in terms of training speed and convergence speed, the improved MS-DDQN method is introduced to incorporate the final return value into the previous steps. Finally, the pre-training learning model is used as the starting point for the second learning model to simulate the UCAV aerial combat decision-making process based on the basic training method, which helps to shorten the training time and improve the learning efficiency. The improved DRL algorithm significantly accelerates the training speed and estimates the target value more accurately during training, and it can be applied to aerial combat decision-making.