A function-based behavioral modeling method for air combat simulation

ion by proposing multiple levels of cascaded hierarchi-cal structures from the perspective of function,namely,the func-tional decision tree.This method is developed to represent behavioral modeling of air combat systems,and its metamodel,execution mechanism,and code generation can provide a sound basis for function-based behavioral modeling.As a proof of con-cept,an air combat simulation is developed to validate this method and the results show that the fighter Alpha built using the proposed framework provides better performance than that using default scripts.

Mastering air combat game with deep reinforcement learning

Reinforcement learning has been applied to air combat problems in recent years,and the idea of curriculum learning is often used for reinforcement learning,but traditional curriculum learning suffers from the problem of plasticity loss in neural networks.Plasticity loss is the difficulty of learning new knowledge after the network has converged.To this end,we propose a motivational curriculum learning distributed proximal policy optimization(MCLDPPO)algorithm,through which trained agents can significantly outperform the predictive game tree and mainstream reinforcement learning methods.The motivational curriculum learning is designed to help the agent gradually improve its combat ability by observing the agent's unsatisfactory performance and providing appropriate rewards as a guide.Furthermore,a complete tactical maneuver is encapsulated based on the existing air combat knowledge,and through the flexible use of these maneuvers,some tactics beyond human knowledge can be realized.In addition,we designed an interruption mechanism for the agent to increase the frequency of decisionmaking when the agent faces an emergency.When the number of threats received by the agent changes,the current action is interrupted in order to reacquire observations and make decisions again.Using the interruption mechanism can significantly improve the performance of the agent.To simulate actual air combat better,we use digital twin technology to simulate real air battles and propose a parallel battlefield mechanism that can run multiple simulation environments simultaneously,effectively improving data throughput.The experimental results demonstrate that the agent can fully utilize the situational information to make reasonable decisions and provide tactical adaptation in the air combat,verifying the effectiveness of the algorithmic framework proposed in this paper.

Optimal confrontation position selecting games model and its application to one-on-one air combat

In the air combat process,confrontation position is the critical factor to determine the confrontation situation,attack effect and escape probability of UAVs.Therefore,selecting the optimal confrontation position becomes the primary goal of maneuver decision-making.By taking the position as the UAV’s maneuver strategy,this paper constructs the optimal confrontation position selecting games(OCPSGs)model.In the OCPSGs model,the payoff function of each UAV is defined by the difference between the comprehensive advantages of both sides,and the strategy space of each UAV at every step is defined by its accessible space determined by the maneuverability.Then we design the limit approximation of mixed strategy Nash equilibrium(LAMSNQ)algorithm,which provides a method to determine the optimal probability distribution of positions in the strategy space.In the simulation phase,we assume the motions on three directions are independent and the strategy space is a cuboid to simplify the model.Several simulations are performed to verify the feasibility,effectiveness and stability of the algorithm.

Cooperative maneuver decision making for multi-UAV air combat based on incomplete information dynamic game

Cooperative autonomous air combat of multiple unmanned aerial vehicles(UAVs)is one of the main combat modes in future air warfare,which becomes even more complicated with highly changeable situation and uncertain information of the opponents.As such,this paper presents a cooperative decision-making method based on incomplete information dynamic game to generate maneuver strategies for multiple UAVs in air combat.Firstly,a cooperative situation assessment model is presented to measure the overall combat situation.Secondly,an incomplete information dynamic game model is proposed to model the dynamic process of air combat,and a dynamic Bayesian network is designed to infer the tactical intention of the opponent.Then a reinforcement learning framework based on multiagent deep deterministic policy gradient is established to obtain the perfect Bayes-Nash equilibrium solution of the air combat game model.Finally,a series of simulations are conducted to verify the effectiveness of the proposed method,and the simulation results show effective synergies and cooperative tactics.

Autonomous air combat decision-making of UAV based on parallel self-play reinforcement learning

Aiming at addressing the problem of manoeuvring decision-making in UAV air combat,this study establishes a one-to-one air combat model,defines missile attack areas,and uses the non-deterministic policy Soft-Actor-Critic(SAC)algorithm in deep reinforcement learning to construct a decision model to realize the manoeuvring process.At the same time,the complexity of the proposed algorithm is calculated,and the stability of the closed-loop system of air combat decision-making controlled by neural network is analysed by the Lyapunov function.This study defines the UAV air combat process as a gaming process and proposes a Parallel Self-Play training SAC algorithm(PSP-SAC)to improve the generalisation performance of UAV control decisions.Simulation results have shown that the proposed algorithm can realize sample sharing and policy sharing in multiple combat environments and can significantly improve the generalisation ability of the model compared to independent training.

An online physical simulation method for enhanced oil recovery by air injection in shale oil

In order to understand the mechanism of air flooding shale oil, an online physical simulation method for enhanced shale oil recovery by air injection was established by integrating CT scanning and nuclear magnetic resonance(NMR). The development effect of shale oil by air flooding under different depletion pressures, the micro-production characteristics of pore throats with different sizes and the mechanism of shale oil recovery by air flooding were analyzed. The effects of air oxygen content, permeability, gas injection pressure, and fractures on the air flooding effect in shale and crude oil production in pores with different sizes were analyzed. The recovery of shale oil can be greatly improved by injecting air into the depleted shale reservoir, but the oil displacement efficiency and the production degree of different levels of pore throats vary with the injection timing. The higher the air oxygen content and the stronger the low-temperature oxidation, the higher the production degree of pores with different sizes and the higher the shale oil recovery. The higher the permeability and the better the pore throat connectivity, the stronger the fluid flow capacity and the higher the shale oil recovery. As the injection pressure increases, the lower limit of the production degree of pore throats decreases, but gas channeling may occur to cause a premature breakthrough;as a result, the recovery increases and then decreases. Fractures can effectively increase the contact area between gas and crude oil, and increase the air sweep coefficient and matrix oil drainage area by supplying oil to fractures through the matrix, which means that a proper fracturing before air injection can help to improve the oil displacement effect under a reasonable production pressure difference.

Air pressure law of a reservoir constructed in karst sinkholes

Karst sinkholes with natural negative landform provide favorable conditions for the pumped storage reservoir construction for less excavation work.However,the construction of the reservoir would plug the natural karst channels for water and air,which would cause remarkable air pressure in karst channels when the groundwater level fluctuates.A large laboratory simulation test was carried out to study the air pressure variation of a reservoir built on the karst sinkhole.The air pressure in the karst channel and inside the model was monitored during the groundwater rising and falling process.Result showed that the variation of air pressure in the karst channel and the surrounding rock exhibited a high degree of similarity.The air pressure increased rapidly at the initial stage of water level rising,followed by a slight decrease,then the air pressure increased sharply when the water level approached the top of the karst cave.The initial peak of air pressure and the final peak of air pressure were defined,and both air pressure peaks were linearly increasing with the water level rising rate.The negative air pressure was also analyzed during the drainage process,which was linearly correlated with the water level falling rate.The causes of air pressure variation in karst channels of a pumped storage reservoir built on the karst sinkhole were discussed.The initial rapid increase,then slight decrease and final sudden increase of air pressure were controlled by the combined effects of air compression in karst channel and air seepage into the surrounding rock.For the drainage process,the instant negative air pressure and gradual recovering of air pressure were controlled by the combined effects of negative air pressure induced by water level falling and air supply from surrounding rock.This work could provide valuable reference for the reservoir construction in karst area.

基于AirTOp的终端区扇区负荷校准方法研究

管制员工作负荷的评估是扇区运行的一个重要论题,它对提高机场终端区容量值,进而增加航班量和经济收益具有重要意义。结合北京终端区的管制员实际工作负荷评估和校准工作,提出了一种基于AirTOp仿真软件的终端区进近扇区负荷校准的新思路和方法。通过对比同等运行环境下的实际管制负荷与仿真输出,校准管制负荷权重设置,获得一个可用于仿真管制员负荷评估的负荷权值汇总表,使用交叉验证方法将校准后的权值作为输入进行面向扇区的计算机仿真。结果显示仿真负荷与实际负荷偏差值小于±2,证明该校准方法可以适用于对复杂终端区扇区负荷的评估。

Analysis of the Erosion-Corrosion Mechanism of the Air Cooler in a Hydrocracking Unit:A Numerical and Experimental Study

Corrosion leakages often occur in the air cooler of a hydrocracking unit,with the failure sites mainly located in the entrance area of the tubes.An analysis of the macroscopic morphology and corrosion products confirmed that the damage was caused by erosion-corrosion(E-C).Numerical and experimental methods were applied to investigate the E-C mechanism in the air cooler.Computational fluid dynamics(CFD)was used to calculate the hydrodynamic parameters of the air cooler.The results showed that there was a biased flow in the air cooler,which led to a significant increase in velocity,turbulent kinetic energy and wall shear within 0.2 m of the tube entrance.A visualization experiment was then performed to determine the principles of migration and transformation of multiphase flow in the air cooler tubes.Various flow patterns(pure droplet flow,mist flow,and annular flow)and their evolutionary processes were clearly depicted experimentally.The initiation mechanism and processes leading to the development of E-C in the air cooler were also determined.This study provided a comprehensive explanation for the E-C failures that occur in air coolers during operation.

Service-oriented air combat simulation architecture

Service-oriented air combat simulation architecture is proposed.The core design goal is high agility which represents the ability to accommodate the simulation requirements change.Its main idea is to design model units as services that can communicate and interoperate with any other services at runtime.A service is autonomous and is fully defined by a description contract which contains some combination of syntactic,semantic,and behavioral information.Based on the architecture,air combat simulation system can be described as an abstract composition of description contracts.It becomes concrete at run time as services that implement the constituent description contracts are discovered and bind.The whole process is a continuous run-time activity that responds to simulation needs and the availability of services.This provides benefits of implementation transparency and minimal dependency between models.Thus,simulation system can minimize the impact of change on it and increase the overall efficiency to respond to requirements change.

Autonomous air combat maneuver decision using Bayesian inference and moving horizon optimization

To reach a higher level of autonomy for unmanned combat aerial vehicle(UCAV) in air combat games, this paper builds an autonomous maneuver decision system. In this system,the air combat game is regarded as a Markov process, so that the air combat situation can be effectively calculated via Bayesian inference theory. According to the situation assessment result,adaptively adjusts the weights of maneuver decision factors, which makes the objective function more reasonable and ensures the superiority situation for UCAV. As the air combat game is characterized by highly dynamic and a significant amount of uncertainty,to enhance the robustness and effectiveness of maneuver decision results, fuzzy logic is used to build the functions of four maneuver decision factors. Accuracy prediction of opponent aircraft is also essential to ensure making a good decision; therefore, a prediction model of opponent aircraft is designed based on the elementary maneuver method. Finally, the moving horizon optimization strategy is used to effectively model the whole air combat maneuver decision process. Various simulations are performed on typical scenario test and close-in dogfight, the results sufficiently demonstrate the superiority of the designed maneuver decision method.

Situation assessment for air combat based on novel semi-supervised naive Bayes

A method is proposed to resolve the typical problem of air combat situation assessment. Taking the one-to-one air combat as an example and on the basis of air combat data recorded by the air combat maneuvering instrument, the problem of air combat situation assessment is equivalent to the situation classification problem of air combat data. The fuzzy C-means clustering algorithm is proposed to cluster the selected air combat sample data and the situation classification of the data is determined by the data correlation analysis in combination with the clustering results and the pilots＇ description of the air combat process. On the basis of semi-supervised naive Bayes classifier, an improved algorithm is proposed based on data classification confidence, through which the situation classification of air combat data is carried out. The simulation results show that the improved algorithm can assess the air combat situation effectively and the improvement of the algorithm can promote the classification performance without significantly affecting the efficiency of the classifier.

UAV cooperative air combat maneuver decision based on multi-agent reinforcement learning

In order to improve the autonomous ability of unmanned aerial vehicles(UAV)to implement air combat mission,many artificial intelligence-based autonomous air combat maneuver decision-making studies have been carried out,but these studies are often aimed at individual decision-making in 1 v1 scenarios which rarely happen in actual air combat.Based on the research of the 1 v1 autonomous air combat maneuver decision,this paper builds a multi-UAV cooperative air combat maneuver decision model based on multi-agent reinforcement learning.Firstly,a bidirectional recurrent neural network(BRNN)is used to achieve communication between UAV individuals,and the multi-UAV cooperative air combat maneuver decision model under the actor-critic architecture is established.Secondly,through combining with target allocation and air combat situation assessment,the tactical goal of the formation is merged with the reinforcement learning goal of every UAV,and a cooperative tactical maneuver policy is generated.The simulation results prove that the multi-UAV cooperative air combat maneuver decision model established in this paper can obtain the cooperative maneuver policy through reinforcement learning,the cooperative maneuver policy can guide UAVs to obtain the overall situational advantage and defeat the opponents under tactical cooperation.

Surface Air Temperature Simulations over China with CMIP5 and CMIP3

Historical simulations of annual mean surface air temperature over China with 25 CMIP5 models were assessed.The observational data from CRUT3v and CN05 were used and further compared with historical simulations of CMIP3.The results show that CMIP5 models were able to simulate the observed warming over China from 1906 to 2005(0.84 C per 100 years)with a warming rate of 0.77 C per 100 years based on the multi-model ensemble(MME).The simulations of surface air temperature in the late 20th century were much better than those in the early 20th century,when only two models could reproduce the extreme warming in the 1940s.The simulations for the spatial distribution of the 20-yearmean(1986–2005)surface air temperature over China fit relatively well with the observations.However,underestimations in surface air temperature climatology were still found almost all over China,and the largest cold bias and simulation uncertainty were found in western China.On sub-regional scale,northern China experienced stronger warming than southern China during 1961–1999,for which the CMIP5 MME provided better simulations.With CMIP5 the diference of warming trends in northern and southern China was underestimated.In general,the CMIP5 simulations are obviously improved in comparison with the CMIP3 simulations in terms of the variation in regional mean surface air temperature,the spatial distribution of surface air temperature climatology and the linear trends in surface air temperature all over China.

A Predator-prey Particle Swarm Optimization Approach to Multiple UCAV Air Combat Modeled by Dynamic Game Theory

Dynamic game theory has received considerable attention as a promising technique for formulating control actions for agents in an extended complex enterprise that involves an adversary. At each decision making step, each side seeks the best scheme with the purpose of maximizing its own objective function. In this paper, a game theoretic approach based on predatorprey particle swarm optimization(PP-PSO) is presented, and the dynamic task assignment problem for multiple unmanned combat aerial vehicles(UCAVs) in military operation is decomposed and modeled as a two-player game at each decision stage. The optimal assignment scheme of each stage is regarded as a mixed Nash equilibrium, which can be solved by using the PP-PSO. The effectiveness of our proposed methodology is verified by a typical example of an air military operation that involves two opposing forces: the attacking force Red and the defense force Blue.

Sequential maneuvering decisions based on multi-stage influence diagram in air combat

A multi-stage influence diagram is used to model the pilot＇s sequential decision making in one on one air combat. The model based on the multi-stage influence diagram graphically describes the elements of decision process, and contains a point-mass model for the dynamics of an aircraft and takes into account the decision maker＇s preferences under uncertain conditions. Considering an active opponent, the opponent＇s maneuvers can be modeled stochastically. The solution of multistage influence diagram can be obtained by converting the multistage influence diagram into a two-level optimization problem. The simulation results show the model is effective.

Numerical Simulation of High Temperature Air Combustion Flames Properties

High temperature air combustion (HTAC) is an attractive technology of saving energy and controlling environment. The mathematical models of turbulent jet flame under the highly preheated air combustion condition are conducted in the paper. The mixture fraction/probability density function model is employed. The results show that the maximum flame temperature is decreased, the temperature in the HTAC furnace is more uniform than that in the conventional furnace, and the NO x emission is low. The numerical results are partially validated by some experimental measurements.

Approach to WTA in air combat using IAFSA-IHS algorithm

In this paper, a static weapon target assignment（WTA）problem is studied. As a critical problem in cooperative air combat,outcome of WTA directly influences the battle. Along with the cost of weapons rising rapidly, it is indispensable to design a target assignment model that can ensure minimizing targets survivability and weapons consumption simultaneously. Afterwards an algorithm named as improved artificial fish swarm algorithm-improved harmony search algorithm（IAFSA-IHS） is proposed to solve the problem. The effect of the proposed algorithm is demonstrated in numerical simulations, and results show that it performs positively in searching the optimal solution and solving the WTA problem.

Air combat decision-making of multiple UCAVs based on constraint strategy games

Game theory can be applied to the air combat decision-making problem of multiple unmanned combat air vehicles(UCAVs).However,it is difficult to have satisfactory decision-making results completely relying on air combat situation information,because there is a lot of time-sensitive information in a complex air combat environment.In this paper,a constraint strategy game approach is developed to generate intelligent decision-making for multiple UCAVs in complex air combat environment with air combat situation information and time-sensitive information.Initially,a constraint strategy game is employed to model attack-defense decision-making problem in complex air combat environment.Then,an algorithm is proposed for solving the constraint strategy game based on linear programming and linear inequality(CSG-LL).Finally,an example is given to illustrate the effectiveness of the proposed approach.

Weapon configuration, allocation and route planning with time windows for multiple unmanned combat air vehicles

Unmanned combat air vehicles(UCAVs) mission planning is a fairly complicated global optimum problem. Military attack missions often employ a fleet of UCAVs equipped with weapons to attack a set of known targets. A UCAV can carry different weapons to accomplish different combat missions. Choice of different weapons will have different effects on the final combat effectiveness. This work presents a mixed integer programming model for simultaneous weapon configuration and route planning of UCAVs, which solves the problem optimally using the IBM ILOG CPLEX optimizer for simple missions. This paper develops a heuristic algorithm to handle the medium-scale and large-scale problems. The experiments demonstrate the performance of the heuristic algorithm in solving the medium scale and large scale problems. Moreover, we give suggestions on how to select the most appropriate algorithm to solve different scale problems.