Multi-UAV cooperative maneuver decision-making for pursuitevasion using improved MADRL

Aiming at the problem of multi-UAV pursuit-evasion confrontation, a UAV cooperative maneuver method based on an improved multi-agent deep reinforcement learning(MADRL) is proposed. In this method, an improved Comm Net network based on a communication mechanism is introduced into a deep reinforcement learning algorithm to solve the multi-agent problem. A layer of gated recurrent unit(GRU) is added to the actor-network structure to remember historical environmental states. Subsequently,another GRU is designed as a communication channel in the Comm Net core network layer to refine communication information between UAVs. Finally, the simulation results of the algorithm in two sets of scenarios are given, and the results show that the method has good effectiveness and applicability.

Research on Maneuver Decision-Making of Multi-Agent Adversarial Game in a Random Interference Environment

The strategy evolution process of game players is highly uncertain due to random emergent situations and other external disturbances.This paper investigates the issue of strategy interaction and behavioral decision-making among game players in simulated confrontation scenarios within a random interference environment.It considers the possible risks that random disturbances may pose to the autonomous decision-making of game players,as well as the impact of participants’manipulative behaviors on the state changes of the players.A nonlinear mathematical model is established to describe the strategy decision-making process of the participants in this scenario.Subsequently,the strategy selection interaction relationship,strategy evolution stability,and dynamic decision-making process of the game players are investigated and verified by simulation experiments.The results show that maneuver-related parameters and random environmental interference factors have different effects on the selection and evolutionary speed of the agent’s strategies.Especially in a highly uncertain environment,even small information asymmetry or miscalculation may have a significant impact on decision-making.This also confirms the feasibility and effectiveness of the method proposed in the paper,which can better explain the behavioral decision-making process of the agent in the interaction process.This study provides feasibility analysis ideas and theoretical references for improving multi-agent interactive decision-making and the interpretability of the game system model.

An air combat maneuver pattern extraction based on time series segmentation and clustering analysis

Target maneuver recognition is a prerequisite for air combat situation awareness,trajectory prediction,threat assessment and maneuver decision.To get rid of the dependence of the current target maneuver recognition method on empirical criteria and sample data,and automatically and adaptively complete the task of extracting the target maneuver pattern,in this paper,an air combat maneuver pattern extraction based on time series segmentation and clustering analysis is proposed by combining autoencoder,G-G clustering algorithm and the selective ensemble clustering analysis algorithm.Firstly,the autoencoder is used to extract key features of maneuvering trajectory to remove the impacts of redundant variables and reduce the data dimension;Then,taking the time information into account,the segmentation of Maneuver characteristic time series is realized with the improved FSTS-AEGG algorithm,and a large number of maneuver primitives are extracted;Finally,the maneuver primitives are grouped into some categories by using the selective ensemble multiple time series clustering algorithm,which can prove that each class represents a maneuver action.The maneuver pattern extraction method is applied to small scale air combat trajectory and can recognize and correctly partition at least 71.3%of maneuver actions,indicating that the method is effective and satisfies the requirements for engineering accuracy.In addition,this method can provide data support for various target maneuvering recognition methods proposed in the literature,greatly reduce the workload and improve the recognition accuracy.

Cooperative guidance law based on super-twisting observer for target maneuvering

To solve the problem that multiple missiles should simultaneously attack unmeasurable maneuvering targets,a guidance law with temporal consistency constraint based on the super-twisting observer is proposed.Firstly,the relative motion equations between multiple missiles and targets are established,and the topological model among multiple agents is considered.Secondly,based on the temporal consistency constraint,a cooperative guidance law for simultaneous arrival with finite-time convergence is derived.Finally,the unknown target maneuver-ing is regarded as bounded interference.Based on the second-order sliding mode theory,a super-twisting sliding mode observer is devised to observe and track the bounded interfer-ence,and the stability of the observer is proved.Compared with the existing research,this approach only needs to obtain the sliding mode variable which simplifies the design process.The simulation results show that the designed cooperative guidance law for maneuvering targets achieves the expected effect.It ensures successful cooperative attacks,even when confronted with strong maneuvering targets.

UAV maneuvering decision-making algorithm based on deep reinforcement learning under the guidance of expert experience

Autonomous umanned aerial vehicle(UAV) manipulation is necessary for the defense department to execute tactical missions given by commanders in the future unmanned battlefield. A large amount of research has been devoted to improving the autonomous decision-making ability of UAV in an interactive environment, where finding the optimal maneuvering decisionmaking policy became one of the key issues for enabling the intelligence of UAV. In this paper, we propose a maneuvering decision-making algorithm for autonomous air-delivery based on deep reinforcement learning under the guidance of expert experience. Specifically, we refine the guidance towards area and guidance towards specific point tasks for the air-delivery process based on the traditional air-to-surface fire control methods.Moreover, we construct the UAV maneuvering decision-making model based on Markov decision processes(MDPs). Specifically, we present a reward shaping method for the guidance towards area and guidance towards specific point tasks using potential-based function and expert-guided advice. The proposed algorithm could accelerate the convergence of the maneuvering decision-making policy and increase the stability of the policy in terms of the output during the later stage of training process. The effectiveness of the proposed maneuvering decision-making policy is illustrated by the curves of training parameters and extensive experimental results for testing the trained policy.

Multiple model PHD filter for tracking sharply maneuvering targets using recursive RANSAC based adaptive birth estimation

An algorithm to track multiple sharply maneuvering targets without prior knowledge about new target birth is proposed. These targets are capable of achieving sharp maneuvers within a short period of time, such as drones and agile missiles.The probability hypothesis density (PHD) filter, which propagates only the first-order statistical moment of the full target posterior, has been shown to be a computationally efficient solution to multitarget tracking problems. However, the standard PHD filter operates on the single dynamic model and requires prior information about target birth distribution, which leads to many limitations in terms of practical applications. In this paper,we introduce a nonzero mean, white noise turn rate dynamic model and generalize jump Markov systems to multitarget case to accommodate sharply maneuvering dynamics. Moreover, to adaptively estimate newborn targets’information, a measurement-driven method based on the recursive random sampling consensus (RANSAC) algorithm is proposed. Simulation results demonstrate that the proposed method achieves significant improvement in tracking multiple sharply maneuvering targets with adaptive birth estimation.

Aerodynamic, Kinematic and Control Coupling Simulation of Aircraft Maneuvering Flight

Aiming at the high angle of attack pull-up and multi-channel roll pull-up coupling problems of high maneuvering aircraft, this paper establishes the flight attitude control rate by means of unsteady flow numerical solution, dynamic unstructured nested mesh assembly method and numerical solution method of flight mechanics equation. On this basis, a virtual flight simulation platform integrating pneumatics, motion and control is established. Based on this virtual flight simulation platform, F-16 aircraft is simulated by high angle of attack pull-up flight mode and multi-channel roll pull-up coupling flight mode. Finally, the influence of rudder on the yaw control channel is investigated. The results show that the numerical virtual flight simulation platform established in this paper has the ability to simulate maneuvering flight of aircraft.

Trajectory and Correction Maneuver During the Transfer from Earth to Halo Orbit

This article addresses the design of the trajectory transferring from Earth to Halo orbit, and proposes a timing closed-loop strategy of correction maneuver during the transfer in the frame of circular restricted three body problem （CR3BP）. The relation between the Floquet multipliers and the magnitudes of Halo orbit is established, so that the suitable magnitude for the aerospace mission is chosen in terms of the stability of Halo orbit. The stable manifold is investigated from the Poincar6 mapping defined which is different from the previous researches, and six types of single-impulse transfer trajectories are attained from the geometry of the invariant manifolds. Based on one of the trajectories of indirect transfer which are ignored in the most of literatures, the stochastic control theory for imperfect information of the discrete linear stochastic system is applied to design the trajectory correction maneuver. The statistical dispersion analysis is performed by Monte-Carlo simulation,

Fish maneuver analysis system based on sequence images

The aim is to establish an automatic system to analyze the maneuver performance of fish. A high speed camera （1 000 frame/s） is employed to record fast-start maneuver. Three steps are taken to analyze the kinematics： first, the midline in the first image is partitioned into equal interval lengths and the coordinates of all inter segmental points are saved. Secondly, these points coordinates are searched in the next frame with the digital image correlation （DIC） method, then these points are fitted with a spline curve function. Repeat this step until all the midlines are figured out frame by frame. Finally, according to the variety of midlines, the kinematics of the fast-start is calculated. Using this system to test carp C-start, the duration is divided into two stages： stage 1 is defined as the formation of the C shape and stage 2 as the return flip of the tail followed with forward motion. By tracing the middle line, the kinematic parameters of turning rate, centre of mass （CM） turning rate, CM turning radius, etc. are obtained.

An Effective Multiple Model Least Squares Method in Tracking of a Maneuvering Target

A polynomial model, time origin shifting model(TOSM, is used to describe the trajectory of a moving target .Based on TOSM, a recursive laeast squares(RLS) algorithm with varied forgetting factor is derived for tracking of a non-maneuvering target. In order to apply this algorithm to maneuvering targets tracking ,a tracking signal is performed on-line to determine what kind of TOSm will be in effect to track a target with different dynamics. An effective multiple model least squares filtering and forecasting method dadpted to real tracking of a maneuvering target is formulated. The algorithm is computationally more effcient than Kalman filter and the percentage improvement from simulations show both of them are considerably alike to some extent.

A new approach to the surgical treatment of parasitic cysts of the liver:Hepatectomy using the liver hanging maneuver

AIM： To review 11 patients with parasitic cysts of the liver, who were treated by hepatic Iobectomy using the liver hanging maneuver （LHM）.METHODS： Between January 2003 and June 2006, we retrospectively analyzed patients who underwent surgical treatment due to parasitic cysts of the liver, at the Ege University School of Medicine, Department of General Surgery. Of these, the patients who underwent hepatic lobectomy using the LHM were reviewed and evaluated for surgical treatment outcome.RESULTS： Over a three-year period, there were 102 patients who underwent surgical treatment for parasitic cysts of the liver. Of these, 11 （10%） patients with parasitic cysts of the liver underwent hepatic Iobectomy using the LHM. Presenting symptoms were abdominal pain, dyspepsia, and cholangitis. Cyst locations were as follows： right lobe filled with cyst, 7 （63%）; segmental location, 2 （18%）; and multiple locations, 2 patients （18%）. All patients underwent hepatic Iobectomy with an anterior approach using the LHM. The intraoperative blood transfusion requirement was one unit for 3 patients and two units for one patient. Postoperative complications included pulmonary atelectasy （2, 18%） and pleural effusion （2, 18%）. No significant morbidity or mortality was observed.CONCLUSION： We concluded that hepatic Iobectomy using the LHM should be considered, not only for hepatic tumors or donor hepatectomy, but also to treat parasitic cysts of the liver.

An approach to improving maneuver performance of coning algorithm

Aiming to improve the maneuver performance of the strapdown inertial navigation attitude coning algorithm a new coning correction structure is constructed by adding a sample to the traditional compressed coning correction structure. According to the given definition of classical coning motion the residual coning correction error based on the new coning correction structure is derived. On the basis of the new structure the frequency Taylor series method is used for designing a coning correction structure coefficient and then a new coning algorithm is obtained.Two types of error models are defined for the coning algorithm performance evaluation under coning environments and maneuver environments respectively.Simulation results indicate that the maneuver accuracy of the new 4-sample coning algorithm is almost double that of the traditional compressed 4-sample coning algorithm. The new coning algorithm has an improved maneuver performance while maintaining coning performance compared to the traditional compressed coning algorithm.

UNSTEADY AERODYNAMIC MODEL OF ISOLATED ROTORIN LARGE-AMPLITUDE MANEUVERS

To improve the rotor off-axis response prediction, aerodynamic models must include the wake distortion effects of the maneuvering rotor. And the crux of the matter is to obtain a precise wake curvature parameter KR. A Peters-He finite-state wake model is improved to incorporate the operating-state-dependent KR to embody maneuver-induced effects. The curvature parameter KR varies with rotor forward speed, thrust and maneuvering angular rate according to a smoking experiment. Moreover, aerodynamic force/moment experiment indicates that after a quasi-step angular input, both on-axis and off-axis rotor responses show that an overshoot and its amplitude increases with the pitching rate. The comparison between theoretical and experimental results shows that the operating-state-accurate curvature parameter must be adopted to obtain accurate aerodynamic forces/moments, especially for the off-axis response. Additionally, combined with a dynamic wake distortion model, the obtained correlation agrees well with experimental data.

Maneuvering target tracking using threshold interacting multiple model algorithm

To avoid missing track caused by the target maneuvers in automatic target tracking system, a new maneuvering target tracking technique called threshold interacting multiple model （TIMM） is proposed. This algorithm is based on the interacting multiple model （IMM） method and applies a threshold controller to improve tracking accuracy. It is also applicable to other advanced algorithms of IMM. In this research, we also compare the position and velocity root mean square （RMS） errors of TIMM and IMM algorithms with two different examples. Simulation results show that the TIMM algorithm is superior to the traditional IMM alzorithm in estimation accuracy.

Autonomous air combat maneuver decision using Bayesian inference and moving horizon optimization

To reach a higher level of autonomy for unmanned combat aerial vehicle(UCAV) in air combat games, this paper builds an autonomous maneuver decision system. In this system,the air combat game is regarded as a Markov process, so that the air combat situation can be effectively calculated via Bayesian inference theory. According to the situation assessment result,adaptively adjusts the weights of maneuver decision factors, which makes the objective function more reasonable and ensures the superiority situation for UCAV. As the air combat game is characterized by highly dynamic and a significant amount of uncertainty,to enhance the robustness and effectiveness of maneuver decision results, fuzzy logic is used to build the functions of four maneuver decision factors. Accuracy prediction of opponent aircraft is also essential to ensure making a good decision; therefore, a prediction model of opponent aircraft is designed based on the elementary maneuver method. Finally, the moving horizon optimization strategy is used to effectively model the whole air combat maneuver decision process. Various simulations are performed on typical scenario test and close-in dogfight, the results sufficiently demonstrate the superiority of the designed maneuver decision method.

Study on Intelligent Control and 3D Real-time Distributed Animation Simulation for Super-maneuver Attack of the New Generation Fighter

The super-maneuver flight performance has a very high tactical value, and the development of this tactical value has great significance. A discussion is devoted to the study of intelligent control methods and technologies of real-time distributed 3-dimensional animation simulation for the super-maneuverable attack of new generational fighter in this paper. A flight control system of super-maneuver is reconstructed by adopting three layers BP neural networks of number 3, and the fire/flight coupler is designed by introducing a fuzzy control rule whose universe of discourse and gain are regulated adaptively on the line. Furthermore, a new method of real-time distributed 3-dimensional animation simulation is put forward, and a real-time distributed 3-dimensional animation simulation tool platform is constructed in this paper. The simulation result is lifelike, perceivable directly and useful.

Variable structure guidance law for attacking surface maneuver targets

The characteristics of surface maneuver targets are analyzed and a 3-D relative motion model for missiles and targets is established. A variable structure guidance law is designed considering the characteristics of targets. In the guidance law, the distance between missiles and targets as well as the missile-target relative velocity are all substituted by estimation values. The estimation errors, the target＇s velocity, and the maneuver acceleration are all treated as bounded disturbance. The guidance law proposed can be implemented conveniently in engineering with little target information. The performance of the guidance system is analyzed theoretically and the numerical simulation result shows the effectiveness of the guidance law.

UAV cooperative air combat maneuver decision based on multi-agent reinforcement learning

In order to improve the autonomous ability of unmanned aerial vehicles(UAV)to implement air combat mission,many artificial intelligence-based autonomous air combat maneuver decision-making studies have been carried out,but these studies are often aimed at individual decision-making in 1 v1 scenarios which rarely happen in actual air combat.Based on the research of the 1 v1 autonomous air combat maneuver decision,this paper builds a multi-UAV cooperative air combat maneuver decision model based on multi-agent reinforcement learning.Firstly,a bidirectional recurrent neural network(BRNN)is used to achieve communication between UAV individuals,and the multi-UAV cooperative air combat maneuver decision model under the actor-critic architecture is established.Secondly,through combining with target allocation and air combat situation assessment,the tactical goal of the formation is merged with the reinforcement learning goal of every UAV,and a cooperative tactical maneuver policy is generated.The simulation results prove that the multi-UAV cooperative air combat maneuver decision model established in this paper can obtain the cooperative maneuver policy through reinforcement learning,the cooperative maneuver policy can guide UAVs to obtain the overall situational advantage and defeat the opponents under tactical cooperation.

Erythropoietin ameliorates early ischemia-reperfusion injury following the Pringle maneuver

AIM:To investigate the protective effect of erythropoietin (Epo) against ischemia-reperfusion injury (IR/I) following the Pringle maneuver (PM),in comparison with conventional steroid administration in a prospective randomized trial. METHODS:Patients were randomized by age, sex, diagnosis, and surgical method, and assigned to three groups:(1) A steroid group (STRD, n= 9) who received 100 mg of hydrocortisone before PM, and on postoperative days 1, 2 and 3, followed by tapering until postoperative day 7; (2) An EPO1 group (n=10) who received 30 000 U of Epo before the PM and at the end of surgery; and (3) An EPO2 group (n=8) who received 60 000 U of Epo before the PM. Hemoglobin (Hb), hematocrit (Ht), aspartate aminotransferase (AST), alanine transaminase (ALT),lactate dehydrogenase (LDH), lactate, interleukin-6 (IL-6),and tumor necrosis factor(TNF)-α were measured before and just after (Day 0) surgery, and on postoperative days 1, 3, 7 and 14. RESULTS: There were no increases in Hb and Ht in the EPO1 and EPO2 groups. AST was signif icantly lower in EPO1 than in STRD on Day 0 (P=0.041), and lower in EPO1 than in STRD and EPO2 on Day 1 (P=0.018). ALT was signif icantly lower in EPO1 than in STRD and EPO2 on Day 0 (P=0.020) and Day 1 (P=0.004). There were no signif icant inter-group differences in the levels of LDH and lactate. IL-6 was signif icantly lower in EPO1 than in STRD and EPO2 on Day 0 (P=0.0036) and Day 1 (P=0.0451). TNF-α was signif icantly lower in EPO1 than in STRD and EPO2 on Day 0 (P=0.0006) and Day 1 (P<0.0001). Furthermore, hospitalization was signif icantly shorter in EPO1 and EPO2 than in STRD.CONCLUSION:Epo has greater potential than steroids to ameliorate IR/I after the PM. Epo at a dose of 30000 U, administered before PM and just after surgery, yields better results.

Wing kinematics measurement and aerodynamics of free-flight maneuvers in drone-flies

The time courses of wing and body kinematics of two free-flying drone-flies, as they performed saccades, were measured using 3D high-speed video, and the morpho- logical parameters of the wings and body of the insects were also measured. The measured wing kinematics was used in a Navier-Stokes solver to compute the aerodynamic forces and moments acting on the insects. The main results are as following. （1） The turn is mainly a 90° change of heading. It is made in about 10 wingbeats （about 55 ms）. It is of interest to note that the number of wingbeats taken to make the turn is approximately the same as and the turning time is only a little different from that of fruitflies measured recently by the same approach, even if the weight of the droneflies is more than 100 times larger than that of the fruitflies. The long axis of body is about 40° from the horizontal during the maneuver. （2） Although the body rotation is mainly about a vertical axis, a relatively large moment around the yaw axis （axis perpendicular to the long axis of body）, called as yaw moment, is mainly needed for the turn, because moment of inertial of the body about the yaw axis is much larger than that about the long axis. （3） The yaw moment is mainly pro- duced by changes in wing angles of attack： in a right turn, for example, the dronefly lets its right wing to have a rather large angle of attack in the downstroke （generally larger than 50°） and a small one in the upstroke to start the turn, and lets its left wing to do so to stop the turn, unlike the fruitflies who generate the yaw moment mainly by changes in the stroke plane and stroke amplitude.