Multi-UAV cooperative maneuver decision-making for pursuitevasion using improved MADRL

Aiming at the problem of multi-UAV pursuit-evasion confrontation, a UAV cooperative maneuver method based on an improved multi-agent deep reinforcement learning(MADRL) is proposed. In this method, an improved Comm Net network based on a communication mechanism is introduced into a deep reinforcement learning algorithm to solve the multi-agent problem. A layer of gated recurrent unit(GRU) is added to the actor-network structure to remember historical environmental states. Subsequently,another GRU is designed as a communication channel in the Comm Net core network layer to refine communication information between UAVs. Finally, the simulation results of the algorithm in two sets of scenarios are given, and the results show that the method has good effectiveness and applicability.

Cooperative maneuver decision making for multi-UAV air combat based on incomplete information dynamic game

Cooperative autonomous air combat of multiple unmanned aerial vehicles(UAVs)is one of the main combat modes in future air warfare,which becomes even more complicated with highly changeable situation and uncertain information of the opponents.As such,this paper presents a cooperative decision-making method based on incomplete information dynamic game to generate maneuver strategies for multiple UAVs in air combat.Firstly,a cooperative situation assessment model is presented to measure the overall combat situation.Secondly,an incomplete information dynamic game model is proposed to model the dynamic process of air combat,and a dynamic Bayesian network is designed to infer the tactical intention of the opponent.Then a reinforcement learning framework based on multiagent deep deterministic policy gradient is established to obtain the perfect Bayes-Nash equilibrium solution of the air combat game model.Finally,a series of simulations are conducted to verify the effectiveness of the proposed method,and the simulation results show effective synergies and cooperative tactics.

Optimal maneuver strategy to improve the observability of angles-only rendezvous

This paper proposes an optimal maneuver strategy to improve the observability of angles-only rendezvous from the perspective of relative navigation.A set of dimensionless relative orbital elements(ROEs)is used to parameterize the relative motion,and the objective function of the observability of anglesonly navigation is established.An analytical solution of the optimal maneuver strategy to improve the observability of anglesonly navigation is obtained by means of numerical analysis.A set of dedicated semi-physical simulation system is built to test the performances of the proposed optimal maneuver strategy.Finally,the effectiveness of the method proposed in this paper is verified through the comparative analysis of the objective function of the observability of angles-only navigation and the performances of the angles-only navigation filter under different maneuver schemes.Compared with the cases without orbital maneuver,it is concluded that the tangential filtering accuracy with the optimal orbital maneuver at the terminal time is increased by 35%on average,and the radial and normal filtering accuracy is increased by 30%on average.

A two-step method to detect broadcast ephemeris unavailable periods caused by BeiDou satellite orbital maneuvers

The space constellation of the BeiDou navigation satellite system(BDS) is a hybrid constellation containing medium earth orbit(MEO) satellites, geostationary earth orbit(GEO) satellites, and inclined geosynchronous orbit(IGSO) satellites. Due to the geosynchronous characteristics of GEO and IGSO, GEO satellites and IGSO satellites often need to perform orbital maneuvers, which can affect the signal-inspace(SIS) availability performance of BeiDou satellites. A two-step detection method for BeiDou satellite orbital maneuvers has been proposed in this paper. The first step is to identify orbital maneuvers based on time series analysis of broadcast ephemeris, and the second step is to verify orbital maneuvers based on bidirectional orbit prediction. The two-step detection method was used to detect the orbital maneuvers of BeiDou satellites in 2019. Through the double guarantees of identification and verification,the detection accuracy of BeiDou satellite orbital maneuvers has been effectively improved. And the orbital maneuver detection results are continued to be used to assess the SIS availability of BeiDou satellites. The results show that the availability loss of GEO satellite orbital maneuvers is about 0.45%-1.07%, and the availability loss of IGSO satellite orbital maneuvers is about 0.12%-0.19%.

Optimal maneuvering strategy of spacecraft evasion based on angles-only measurement and observability analysis

Spacecraft orbit evasion is an effective method to ensure space safety. In the spacecraft’s orbital plane, the space non-cooperate target with autonomous approaching to the spacecraft may have a dangerous rendezvous. To deal with this problem, an optimal maneuvering strategy based on the relative navigation observability degree is proposed with angles-only measurements. A maneuver evasion relative navigation model in the spacecraft’s orbital plane is constructed and the observability measurement criteria with process noise and measurement noise are defined based on the posterior Cramer-Rao lower bound. Further, the optimal maneuver evasion strategy in spacecraft’s orbital plane based on the observability is proposed. The strategy provides a new idea for spacecraft to evade safety threats autonomously. Compared with the spacecraft evasion problem based on the absolute navigation, more accurate evasion results can be obtained. The simulation indicates that this optimal strategy can weaken the system’s observability and reduce the state estimation accuracy of the non-cooperative target, making it impossible for the non-cooperative target to accurately approach the spacecraft.

Air combat target maneuver trajectory prediction based on robust regularized Volterra series and adaptive ensemble online transfer learning

Target maneuver trajectory prediction is an important prerequisite for air combat situation awareness and maneuver decision-making.However,how to use a large amount of trajectory data generated by air combat confrontation training to achieve real-time and accurate prediction of target maneuver trajectory is an urgent problem to be solved.To solve this problem,in this paper,a hybrid algorithm based on transfer learning,online learning,ensemble learning,regularization technology,target maneuvering segmentation point recognition algorithm,and Volterra series,abbreviated as AERTrOS-Volterra is proposed.Firstly,the model makes full use of a large number of trajectory sample data generated by air combat confrontation training,and constructs a Tr-Volterra algorithm framework suitable for air combat target maneuver trajectory prediction,which realizes the extraction of effective information from the historical trajectory data.Secondly,in order to improve the real-time online prediction accuracy and robustness of the prediction model in complex electromagnetic environments,on the basis of the TrVolterra algorithm framework,a robust regularized online Sequential Volterra prediction model is proposed by integrating online learning method,regularization technology and inverse weighting calculation method based on the priori error.Finally,inspired by the preferable performance of models ensemble,ensemble learning scheme is also incorporated into our proposed algorithm,which adaptively updates the ensemble prediction model according to the performance of the model on real-time samples and the recognition results of target maneuvering segmentation points,including the adaptation of model weights;adaptation of parameters;and dynamic inclusion and removal of models.Compared with many existing time series prediction methods,the newly proposed target maneuver trajectory prediction algorithm can fully mine the prior knowledge contained in the historical data to assist the current prediction.The rationality and effectiveness of the proposed algorithm are verified by simulation on three sets of chaotic time series data sets and a set of real target maneuver trajectory data sets.

Impact point prediction guidance of ballistic missile in high maneuver penetration condition

An impact point prediction(IPP) guidance based on supervised learning is proposed to address the problem of precise guidance for the ballistic missile in high maneuver penetration condition.An accurate ballistic trajectory model is applied to generate training samples,and ablation experiments are conducted to determine the mapping relationship between the flight state and the impact point.At the same time,the impact point coordinates are decoupled to improve the prediction accuracy,and the sigmoid activation function is improved to ameliorate the prediction efficiency.Therefore,an IPP neural network model,which solves the contradiction between the accuracy and the speed of the IPP,is established.In view of the performance deviation of the divert control system,the mapping relationship between the guidance parameters and the impact deviation is analysed based on the variational principle.In addition,a fast iterative model of guidance parameters is designed for reference to the Newton iteration method,which solves the nonlinear strong coupling problem of the guidance parameter solution.Monte Carlo simulation results show that the prediction accuracy of the impact point is high,with a 3 σ prediction error of 4.5 m,and the guidance method is robust,with a 3 σ error of 7.5 m.On the STM32F407 singlechip microcomputer,a single IPP takes about 2.374 ms,and a single guidance solution takes about9.936 ms,which has a good real-time performance and a certain engineering application value.

Incidence,characteristics and risk factors for alveolar recruitment maneuver-related hypotension in patients undergoing laparoscopic colorectal cancer resection

BACKGROUND Alveolar recruitment maneuvers(ARMs)may lead to transient hypotension,but the clinical characteristics of this induced hypotension are poorly understood.We investigated the characteristics of ARM-related hypotension in patients who underwent laparoscopic colorectal cancer resection.AIM To investigate the characteristics of ARM-related hypotension in patients who underwent laparoscopic colorectal cancer resection.METHODS This was a secondary analysis of the PROtective Ventilation using Open Lung approach Or Not trial and included 140 subjects.An ARM was repeated every 30 min during intraoperative mechanical ventilation.The primary endpoint was ARM-related hypotension,defined as a mean arterial pressure(MAP)<60 mmHg during an ARM or within 5 min after an ARM.The risk factors for hypotension were identified.The peri-ARM changes in blood pressure were analyzed for the first three ARMs(ARM_(1,2,3))and the last ARM(ARMl_(ast)).RESULTS Thirty-four subjects(24.3%)developed ARM-related hypotension.Of all 1027 ARMs,37(3.61%)induced hypotension.More ARMs under nonpneumoperitoneum(33/349,9.46%)than under pneumoperitoneum conditions(4/678,0.59%)induced hypotension(P<0.01).The incidence of hypotension was higher at ARM_(1)points than at non-ARM_(1)points(18/135,13.3%vs 19/892,2.1%;P<0.01).The median percentage decrease in the MAP at ARM1 was 14%.Age≥74 years,blood loss≥150 mL and peak inspiratory pressure under pneumoperitoneum<24 cm H_(2)O were risk factors for ARM-related hypotension.CONCLUSION When the ARM was repeated intraoperatively,a quarter of subjects developed ARM-related hypotension,but only 3.61%of ARMs induced hypotension.ARM-related hypotension most occurred in a hemodynamically unstable state or a hypovolemic state,and in elderly subjects.Fortunately,ARMs that were performed under pneumoperitoneum conditions had less impact on blood pressure.

Situational continuity-based air combat autonomous maneuvering decision-making

In order to improve the performance of UAV's autonomous maneuvering decision-making,this paper proposes a decision-making method based on situational continuity.The algorithm in this paper designs a situation evaluation function with strong guidance,then trains the Long Short-Term Memory(LSTM)under the framework of Deep Q Network(DQN)for air combat maneuvering decision-making.Considering the continuity between adjacent situations,the method takes multiple consecutive situations as one input of the neural network.To reflect the difference between adjacent situations,the method takes the difference of situation evaluation value as the reward of reinforcement learning.In different scenarios,the algorithm proposed in this paper is compared with the algorithm based on the Fully Neural Network(FNN)and the algorithm based on statistical principles respectively.The results show that,compared with the FNN algorithm,the algorithm proposed in this paper is more accurate and forwardlooking.Compared with the algorithm based on the statistical principles,the decision-making of the algorithm proposed in this paper is more efficient and its real-time performance is better.

CFD Investigations of Ship Maneuvering in Waves Using naoe-FOAM-SJTU Solver

Ship maneuvering in waves includes the performance of ship resistance, seakeeping, propulsion, and maneuverability. It is a complex hydrodynamic problem with the interaction of many factors. With the purpose of directly predicting the behavior of ship maneuvering in waves, a CFD solver named naoe-FOAM-SJTU is developed by the Computational Marine Hydrodynamics Lab(CMHL) in Shanghai Jiao Tong University. The solver is based on open source platform OpenFOAM and has introduced dynamic overset grid technology to handle complex ship hull-propeller-rudder motion system. Maneuvering control module based on feedback control mechanism is also developed to accurately simulate corresponding motion behavior of free running ship maneuver. Inlet boundary wavemaker and relaxation zone technique is used to generate desired waves. Based on the developed modules, unsteady Reynolds-averaged Navier-Stokes(RANS) computations are carried out for several validation cases of free running ship maneuver in waves including zigzag, turning circle, and course keeping maneuvers. The simulation results are compared with available benchmark data. Ship motions, trajectories, and other maneuvering parameters are consistent with available experimental data, which indicate that the present solver can be suitable and reliable in predicting the performance of ship maneuvering in waves. Flow visualizations, such as free surface elevation, wake flow, vortical structures, are presented to explain the hydrodynamic performance of ship maneuvering in waves. Large flow separation can be observed around propellers and rudders. It is concluded that RANS approach is not accurate enough for predicting ship maneuvering in waves with large flow separations and detached eddy simulation(DES) or large eddy simulation(LES) computations are required to improve the prediction accuracy.

Autonomous air combat maneuver decision using Bayesian inference and moving horizon optimization

To reach a higher level of autonomy for unmanned combat aerial vehicle(UCAV) in air combat games, this paper builds an autonomous maneuver decision system. In this system,the air combat game is regarded as a Markov process, so that the air combat situation can be effectively calculated via Bayesian inference theory. According to the situation assessment result,adaptively adjusts the weights of maneuver decision factors, which makes the objective function more reasonable and ensures the superiority situation for UCAV. As the air combat game is characterized by highly dynamic and a significant amount of uncertainty,to enhance the robustness and effectiveness of maneuver decision results, fuzzy logic is used to build the functions of four maneuver decision factors. Accuracy prediction of opponent aircraft is also essential to ensure making a good decision; therefore, a prediction model of opponent aircraft is designed based on the elementary maneuver method. Finally, the moving horizon optimization strategy is used to effectively model the whole air combat maneuver decision process. Various simulations are performed on typical scenario test and close-in dogfight, the results sufficiently demonstrate the superiority of the designed maneuver decision method.

Erythropoietin ameliorates early ischemia-reperfusion injury following the Pringle maneuver

AIM:To investigate the protective effect of erythropoietin (Epo) against ischemia-reperfusion injury (IR/I) following the Pringle maneuver (PM),in comparison with conventional steroid administration in a prospective randomized trial. METHODS:Patients were randomized by age, sex, diagnosis, and surgical method, and assigned to three groups:(1) A steroid group (STRD, n= 9) who received 100 mg of hydrocortisone before PM, and on postoperative days 1, 2 and 3, followed by tapering until postoperative day 7; (2) An EPO1 group (n=10) who received 30 000 U of Epo before the PM and at the end of surgery; and (3) An EPO2 group (n=8) who received 60 000 U of Epo before the PM. Hemoglobin (Hb), hematocrit (Ht), aspartate aminotransferase (AST), alanine transaminase (ALT),lactate dehydrogenase (LDH), lactate, interleukin-6 (IL-6),and tumor necrosis factor(TNF)-α were measured before and just after (Day 0) surgery, and on postoperative days 1, 3, 7 and 14. RESULTS: There were no increases in Hb and Ht in the EPO1 and EPO2 groups. AST was signif icantly lower in EPO1 than in STRD on Day 0 (P=0.041), and lower in EPO1 than in STRD and EPO2 on Day 1 (P=0.018). ALT was signif icantly lower in EPO1 than in STRD and EPO2 on Day 0 (P=0.020) and Day 1 (P=0.004). There were no signif icant inter-group differences in the levels of LDH and lactate. IL-6 was signif icantly lower in EPO1 than in STRD and EPO2 on Day 0 (P=0.0036) and Day 1 (P=0.0451). TNF-α was signif icantly lower in EPO1 than in STRD and EPO2 on Day 0 (P=0.0006) and Day 1 (P<0.0001). Furthermore, hospitalization was signif icantly shorter in EPO1 and EPO2 than in STRD.CONCLUSION:Epo has greater potential than steroids to ameliorate IR/I after the PM. Epo at a dose of 30000 U, administered before PM and just after surgery, yields better results.

Variable structure guidance law for attacking surface maneuver targets

The characteristics of surface maneuver targets are analyzed and a 3-D relative motion model for missiles and targets is established. A variable structure guidance law is designed considering the characteristics of targets. In the guidance law, the distance between missiles and targets as well as the missile-target relative velocity are all substituted by estimation values. The estimation errors, the target＇s velocity, and the maneuver acceleration are all treated as bounded disturbance. The guidance law proposed can be implemented conveniently in engineering with little target information. The performance of the guidance system is analyzed theoretically and the numerical simulation result shows the effectiveness of the guidance law.

Wing kinematics measurement and aerodynamics of free-flight maneuvers in drone-flies

The time courses of wing and body kinematics of two free-flying drone-flies, as they performed saccades, were measured using 3D high-speed video, and the morpho- logical parameters of the wings and body of the insects were also measured. The measured wing kinematics was used in a Navier-Stokes solver to compute the aerodynamic forces and moments acting on the insects. The main results are as following. （1） The turn is mainly a 90° change of heading. It is made in about 10 wingbeats （about 55 ms）. It is of interest to note that the number of wingbeats taken to make the turn is approximately the same as and the turning time is only a little different from that of fruitflies measured recently by the same approach, even if the weight of the droneflies is more than 100 times larger than that of the fruitflies. The long axis of body is about 40° from the horizontal during the maneuver. （2） Although the body rotation is mainly about a vertical axis, a relatively large moment around the yaw axis （axis perpendicular to the long axis of body）, called as yaw moment, is mainly needed for the turn, because moment of inertial of the body about the yaw axis is much larger than that about the long axis. （3） The yaw moment is mainly pro- duced by changes in wing angles of attack： in a right turn, for example, the dronefly lets its right wing to have a rather large angle of attack in the downstroke （generally larger than 50°） and a small one in the upstroke to start the turn, and lets its left wing to do so to stop the turn, unlike the fruitflies who generate the yaw moment mainly by changes in the stroke plane and stroke amplitude.

An impact angle constraint integral sliding mode guidance law for maneuvering targets interception

An integral sliding mode guidance law(ISMGL)combined with the advantages of the integral sliding mode control(SMC)method is designed to address maneuvering target interception problems with impact angle constraints.The relative motion equation of the missile and the target considering the impact angle constraint is established in the longitudinal plane,and an integral sliding mode surface is constructed.The proposed guidance law resolves the existence of a steady-state error problem in the traditional SMC.Such a guidance law ensures that the missile hits the target with an ideal impact angle in finite time and the missile is kept highly robust throughout the interception process.By adopting the dynamic surface control method,the ISMGL is designed considering the impact angle constraints and the autopilot dynamic characteristics.According to the Lyapunov stability theorem,all states of the closed-loop system are finally proven to be uniformly bounded.Simulation results are compared with the general sliding mode guidance law and the trajectory shaping guidance law,and the findings verify the effectiveness and superiority of the ISMGL.

Quick repositioning maneuver for horizontal semicircular canal benign paroxysmal positional vertigo

Objective:To investigate the efficacy of quick repositioning maneuver for horizontal semicircular canal benign paroxysmal positional vertigo (H-BPPV). Methods:Clinical data of 67 patients with H-BPPV who underwent quick repositioning maneuver in our hospital from July 2009 to November 2014 were retrospectively analyzed. The maneuver involved rotating the patient in the axial plane for 180? from the involved side towards contralateral side as quickly as possible. Results:Complete symptom resolution was achieved in 61 patients (91.0%) at one week and in 64 patients (95.5%) at 3 months post-treatment. During the repositioning maneuver process, there were no obvious untoward responses except transient nausea with or without vomiting in a few patients. Conclusion:The results indicate that the quick repositioning maneuver is an easy and effective alternative treatment in the management of H-BPPV. Copyright ? 2015 The Authors. Production & hosting by Elsevier (Singapore) Pte Ltd On behalf of PLA General Hospital Department of Otolaryngology Head and Neck Surgery. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Adaptive Maneuvering Frequency Method of Current Statistical Model

Current statistical model(CSM) has a good performance in maneuvering target tracking. However, the fixed maneuvering frequency will deteriorate the tracking results, such as a serious dynamic delay, a slowly converging speedy and a limited precision when using Kalman filter(KF) algorithm. In this study, a new current statistical model and a new Kalman filter are proposed to improve the performance of maneuvering target tracking. The new model which employs innovation dominated subjection function to adaptively adjust maneuvering frequency has a better performance in step maneuvering target tracking, while a fluctuant phenomenon appears. As far as this problem is concerned, a new adaptive fading Kalman filter is proposed as well. In the new Kalman filter, the prediction values are amended in time by setting judgment and amendment rules,so that tracking precision and fluctuant phenomenon of the new current statistical model are improved. The results of simulation indicate the effectiveness of the new algorithm and the practical guiding significance.

Sequential maneuvering decisions based on multi-stage influence diagram in air combat

A multi-stage influence diagram is used to model the pilot＇s sequential decision making in one on one air combat. The model based on the multi-stage influence diagram graphically describes the elements of decision process, and contains a point-mass model for the dynamics of an aircraft and takes into account the decision maker＇s preferences under uncertain conditions. Considering an active opponent, the opponent＇s maneuvers can be modeled stochastically. The solution of multistage influence diagram can be obtained by converting the multistage influence diagram into a two-level optimization problem. The simulation results show the model is effective.

Chaotic attitude and reorientation maneuver for completely liquid-filled spacecraft with flexible appendage

The present paper investigates the chaotic attitude dynamics and reorientation maneuver for completely viscous liquid-filled spacecraft with flexible appendage. All of the equations of motion are derived by using Lagrangian mechanics and then transformed into a form consisting of an unperturbed part plus perturbed terms so that the system＇s nonlinear characteristics can be exploited in phase space. Emphases are laid on the chaotic attitude dynamics produced from certain sets of physical parameter values of the spacecraft when energy dissipation acts to derive the body from minor to major axis spin. Numerical solutions of these equations show that the attitude dynamics of liquid-filled flexible spacecraft possesses characteristics common to random, non- periodic solutions and chaos, and it is demonstrated that the desired reorientation maneuver is guaranteed by using a pair of thruster impulses. The control strategy for reorientation maneuver is designed and the numerical simulation results are presented for both the uncontrolled and controlled spins transition.

NONLINEAR INVERSE SIMULATION FOR THE MANEUVERING FLIGHT OF COAXIAL ROTOR HELICOPTERS

The maneuvering flight governing equations for coaxial rotor helicopters are established. By introducing induced velocity interference factor analysis, the coaxial rotor aerodynamic interference can be taken into account. With the combination of coaxial rotor helicopter control features and nonlinear inverse solution technique, the governing equations for maneuvering flight can be solved so as to determine helicopter control input, control force and moment, and helicopter body attitudes which are needed for performing the defined maneuver. Finally, as an example of this methods engineering application, the calculated results with level turn, lateral jink maneuvers are presented and simply analyzed.