Multi-UAV cooperative maneuver decision-making for pursuitevasion using improved MADRL

Aiming at the problem of multi-UAV pursuit-evasion confrontation, a UAV cooperative maneuver method based on an improved multi-agent deep reinforcement learning(MADRL) is proposed. In this method, an improved Comm Net network based on a communication mechanism is introduced into a deep reinforcement learning algorithm to solve the multi-agent problem. A layer of gated recurrent unit(GRU) is added to the actor-network structure to remember historical environmental states. Subsequently,another GRU is designed as a communication channel in the Comm Net core network layer to refine communication information between UAVs. Finally, the simulation results of the algorithm in two sets of scenarios are given, and the results show that the method has good effectiveness and applicability.

An air combat maneuver pattern extraction based on time series segmentation and clustering analysis

Target maneuver recognition is a prerequisite for air combat situation awareness,trajectory prediction,threat assessment and maneuver decision.To get rid of the dependence of the current target maneuver recognition method on empirical criteria and sample data,and automatically and adaptively complete the task of extracting the target maneuver pattern,in this paper,an air combat maneuver pattern extraction based on time series segmentation and clustering analysis is proposed by combining autoencoder,G-G clustering algorithm and the selective ensemble clustering analysis algorithm.Firstly,the autoencoder is used to extract key features of maneuvering trajectory to remove the impacts of redundant variables and reduce the data dimension;Then,taking the time information into account,the segmentation of Maneuver characteristic time series is realized with the improved FSTS-AEGG algorithm,and a large number of maneuver primitives are extracted;Finally,the maneuver primitives are grouped into some categories by using the selective ensemble multiple time series clustering algorithm,which can prove that each class represents a maneuver action.The maneuver pattern extraction method is applied to small scale air combat trajectory and can recognize and correctly partition at least 71.3%of maneuver actions,indicating that the method is effective and satisfies the requirements for engineering accuracy.In addition,this method can provide data support for various target maneuvering recognition methods proposed in the literature,greatly reduce the workload and improve the recognition accuracy.

Research on Maneuver Decision-Making of Multi-Agent Adversarial Game in a Random Interference Environment

The strategy evolution process of game players is highly uncertain due to random emergent situations and other external disturbances.This paper investigates the issue of strategy interaction and behavioral decision-making among game players in simulated confrontation scenarios within a random interference environment.It considers the possible risks that random disturbances may pose to the autonomous decision-making of game players,as well as the impact of participants’manipulative behaviors on the state changes of the players.A nonlinear mathematical model is established to describe the strategy decision-making process of the participants in this scenario.Subsequently,the strategy selection interaction relationship,strategy evolution stability,and dynamic decision-making process of the game players are investigated and verified by simulation experiments.The results show that maneuver-related parameters and random environmental interference factors have different effects on the selection and evolutionary speed of the agent’s strategies.Especially in a highly uncertain environment,even small information asymmetry or miscalculation may have a significant impact on decision-making.This also confirms the feasibility and effectiveness of the method proposed in the paper,which can better explain the behavioral decision-making process of the agent in the interaction process.This study provides feasibility analysis ideas and theoretical references for improving multi-agent interactive decision-making and the interpretability of the game system model.

UAV maneuvering decision-making algorithm based on deep reinforcement learning under the guidance of expert experience

Autonomous umanned aerial vehicle(UAV) manipulation is necessary for the defense department to execute tactical missions given by commanders in the future unmanned battlefield. A large amount of research has been devoted to improving the autonomous decision-making ability of UAV in an interactive environment, where finding the optimal maneuvering decisionmaking policy became one of the key issues for enabling the intelligence of UAV. In this paper, we propose a maneuvering decision-making algorithm for autonomous air-delivery based on deep reinforcement learning under the guidance of expert experience. Specifically, we refine the guidance towards area and guidance towards specific point tasks for the air-delivery process based on the traditional air-to-surface fire control methods.Moreover, we construct the UAV maneuvering decision-making model based on Markov decision processes(MDPs). Specifically, we present a reward shaping method for the guidance towards area and guidance towards specific point tasks using potential-based function and expert-guided advice. The proposed algorithm could accelerate the convergence of the maneuvering decision-making policy and increase the stability of the policy in terms of the output during the later stage of training process. The effectiveness of the proposed maneuvering decision-making policy is illustrated by the curves of training parameters and extensive experimental results for testing the trained policy.

Multiple model PHD filter for tracking sharply maneuvering targets using recursive RANSAC based adaptive birth estimation

An algorithm to track multiple sharply maneuvering targets without prior knowledge about new target birth is proposed. These targets are capable of achieving sharp maneuvers within a short period of time, such as drones and agile missiles.The probability hypothesis density (PHD) filter, which propagates only the first-order statistical moment of the full target posterior, has been shown to be a computationally efficient solution to multitarget tracking problems. However, the standard PHD filter operates on the single dynamic model and requires prior information about target birth distribution, which leads to many limitations in terms of practical applications. In this paper,we introduce a nonzero mean, white noise turn rate dynamic model and generalize jump Markov systems to multitarget case to accommodate sharply maneuvering dynamics. Moreover, to adaptively estimate newborn targets’information, a measurement-driven method based on the recursive random sampling consensus (RANSAC) algorithm is proposed. Simulation results demonstrate that the proposed method achieves significant improvement in tracking multiple sharply maneuvering targets with adaptive birth estimation.

Optimal maneuvering strategy of spacecraft evasion based on angles-only measurement and observability analysis

Spacecraft orbit evasion is an effective method to ensure space safety. In the spacecraft’s orbital plane, the space non-cooperate target with autonomous approaching to the spacecraft may have a dangerous rendezvous. To deal with this problem, an optimal maneuvering strategy based on the relative navigation observability degree is proposed with angles-only measurements. A maneuver evasion relative navigation model in the spacecraft’s orbital plane is constructed and the observability measurement criteria with process noise and measurement noise are defined based on the posterior Cramer-Rao lower bound. Further, the optimal maneuver evasion strategy in spacecraft’s orbital plane based on the observability is proposed. The strategy provides a new idea for spacecraft to evade safety threats autonomously. Compared with the spacecraft evasion problem based on the absolute navigation, more accurate evasion results can be obtained. The simulation indicates that this optimal strategy can weaken the system’s observability and reduce the state estimation accuracy of the non-cooperative target, making it impossible for the non-cooperative target to accurately approach the spacecraft.

Cooperative maneuver decision making for multi-UAV air combat based on incomplete information dynamic game

Cooperative autonomous air combat of multiple unmanned aerial vehicles(UAVs)is one of the main combat modes in future air warfare,which becomes even more complicated with highly changeable situation and uncertain information of the opponents.As such,this paper presents a cooperative decision-making method based on incomplete information dynamic game to generate maneuver strategies for multiple UAVs in air combat.Firstly,a cooperative situation assessment model is presented to measure the overall combat situation.Secondly,an incomplete information dynamic game model is proposed to model the dynamic process of air combat,and a dynamic Bayesian network is designed to infer the tactical intention of the opponent.Then a reinforcement learning framework based on multiagent deep deterministic policy gradient is established to obtain the perfect Bayes-Nash equilibrium solution of the air combat game model.Finally,a series of simulations are conducted to verify the effectiveness of the proposed method,and the simulation results show effective synergies and cooperative tactics.

Optimal maneuver strategy to improve the observability of angles-only rendezvous

This paper proposes an optimal maneuver strategy to improve the observability of angles-only rendezvous from the perspective of relative navigation.A set of dimensionless relative orbital elements(ROEs)is used to parameterize the relative motion,and the objective function of the observability of anglesonly navigation is established.An analytical solution of the optimal maneuver strategy to improve the observability of anglesonly navigation is obtained by means of numerical analysis.A set of dedicated semi-physical simulation system is built to test the performances of the proposed optimal maneuver strategy.Finally,the effectiveness of the method proposed in this paper is verified through the comparative analysis of the objective function of the observability of angles-only navigation and the performances of the angles-only navigation filter under different maneuver schemes.Compared with the cases without orbital maneuver,it is concluded that the tangential filtering accuracy with the optimal orbital maneuver at the terminal time is increased by 35%on average,and the radial and normal filtering accuracy is increased by 30%on average.

Air combat target maneuver trajectory prediction based on robust regularized Volterra series and adaptive ensemble online transfer learning

Target maneuver trajectory prediction is an important prerequisite for air combat situation awareness and maneuver decision-making.However,how to use a large amount of trajectory data generated by air combat confrontation training to achieve real-time and accurate prediction of target maneuver trajectory is an urgent problem to be solved.To solve this problem,in this paper,a hybrid algorithm based on transfer learning,online learning,ensemble learning,regularization technology,target maneuvering segmentation point recognition algorithm,and Volterra series,abbreviated as AERTrOS-Volterra is proposed.Firstly,the model makes full use of a large number of trajectory sample data generated by air combat confrontation training,and constructs a Tr-Volterra algorithm framework suitable for air combat target maneuver trajectory prediction,which realizes the extraction of effective information from the historical trajectory data.Secondly,in order to improve the real-time online prediction accuracy and robustness of the prediction model in complex electromagnetic environments,on the basis of the TrVolterra algorithm framework,a robust regularized online Sequential Volterra prediction model is proposed by integrating online learning method,regularization technology and inverse weighting calculation method based on the priori error.Finally,inspired by the preferable performance of models ensemble,ensemble learning scheme is also incorporated into our proposed algorithm,which adaptively updates the ensemble prediction model according to the performance of the model on real-time samples and the recognition results of target maneuvering segmentation points,including the adaptation of model weights;adaptation of parameters;and dynamic inclusion and removal of models.Compared with many existing time series prediction methods,the newly proposed target maneuver trajectory prediction algorithm can fully mine the prior knowledge contained in the historical data to assist the current prediction.The rationality and effectiveness of the proposed algorithm are verified by simulation on three sets of chaotic time series data sets and a set of real target maneuver trajectory data sets.

Impact point prediction guidance of ballistic missile in high maneuver penetration condition

An impact point prediction(IPP) guidance based on supervised learning is proposed to address the problem of precise guidance for the ballistic missile in high maneuver penetration condition.An accurate ballistic trajectory model is applied to generate training samples,and ablation experiments are conducted to determine the mapping relationship between the flight state and the impact point.At the same time,the impact point coordinates are decoupled to improve the prediction accuracy,and the sigmoid activation function is improved to ameliorate the prediction efficiency.Therefore,an IPP neural network model,which solves the contradiction between the accuracy and the speed of the IPP,is established.In view of the performance deviation of the divert control system,the mapping relationship between the guidance parameters and the impact deviation is analysed based on the variational principle.In addition,a fast iterative model of guidance parameters is designed for reference to the Newton iteration method,which solves the nonlinear strong coupling problem of the guidance parameter solution.Monte Carlo simulation results show that the prediction accuracy of the impact point is high,with a 3 σ prediction error of 4.5 m,and the guidance method is robust,with a 3 σ error of 7.5 m.On the STM32F407 singlechip microcomputer,a single IPP takes about 2.374 ms,and a single guidance solution takes about9.936 ms,which has a good real-time performance and a certain engineering application value.

A two-step method to detect broadcast ephemeris unavailable periods caused by BeiDou satellite orbital maneuvers

The space constellation of the BeiDou navigation satellite system(BDS) is a hybrid constellation containing medium earth orbit(MEO) satellites, geostationary earth orbit(GEO) satellites, and inclined geosynchronous orbit(IGSO) satellites. Due to the geosynchronous characteristics of GEO and IGSO, GEO satellites and IGSO satellites often need to perform orbital maneuvers, which can affect the signal-inspace(SIS) availability performance of BeiDou satellites. A two-step detection method for BeiDou satellite orbital maneuvers has been proposed in this paper. The first step is to identify orbital maneuvers based on time series analysis of broadcast ephemeris, and the second step is to verify orbital maneuvers based on bidirectional orbit prediction. The two-step detection method was used to detect the orbital maneuvers of BeiDou satellites in 2019. Through the double guarantees of identification and verification,the detection accuracy of BeiDou satellite orbital maneuvers has been effectively improved. And the orbital maneuver detection results are continued to be used to assess the SIS availability of BeiDou satellites. The results show that the availability loss of GEO satellite orbital maneuvers is about 0.45%-1.07%, and the availability loss of IGSO satellite orbital maneuvers is about 0.12%-0.19%.

Incidence,characteristics and risk factors for alveolar recruitment maneuver-related hypotension in patients undergoing laparoscopic colorectal cancer resection

BACKGROUND Alveolar recruitment maneuvers(ARMs)may lead to transient hypotension,but the clinical characteristics of this induced hypotension are poorly understood.We investigated the characteristics of ARM-related hypotension in patients who underwent laparoscopic colorectal cancer resection.AIM To investigate the characteristics of ARM-related hypotension in patients who underwent laparoscopic colorectal cancer resection.METHODS This was a secondary analysis of the PROtective Ventilation using Open Lung approach Or Not trial and included 140 subjects.An ARM was repeated every 30 min during intraoperative mechanical ventilation.The primary endpoint was ARM-related hypotension,defined as a mean arterial pressure(MAP)<60 mmHg during an ARM or within 5 min after an ARM.The risk factors for hypotension were identified.The peri-ARM changes in blood pressure were analyzed for the first three ARMs(ARM_(1,2,3))and the last ARM(ARMl_(ast)).RESULTS Thirty-four subjects(24.3%)developed ARM-related hypotension.Of all 1027 ARMs,37(3.61%)induced hypotension.More ARMs under nonpneumoperitoneum(33/349,9.46%)than under pneumoperitoneum conditions(4/678,0.59%)induced hypotension(P<0.01).The incidence of hypotension was higher at ARM_(1)points than at non-ARM_(1)points(18/135,13.3%vs 19/892,2.1%;P<0.01).The median percentage decrease in the MAP at ARM1 was 14%.Age≥74 years,blood loss≥150 mL and peak inspiratory pressure under pneumoperitoneum<24 cm H_(2)O were risk factors for ARM-related hypotension.CONCLUSION When the ARM was repeated intraoperatively,a quarter of subjects developed ARM-related hypotension,but only 3.61%of ARMs induced hypotension.ARM-related hypotension most occurred in a hemodynamically unstable state or a hypovolemic state,and in elderly subjects.Fortunately,ARMs that were performed under pneumoperitoneum conditions had less impact on blood pressure.

Situational continuity-based air combat autonomous maneuvering decision-making

In order to improve the performance of UAV's autonomous maneuvering decision-making,this paper proposes a decision-making method based on situational continuity.The algorithm in this paper designs a situation evaluation function with strong guidance,then trains the Long Short-Term Memory(LSTM)under the framework of Deep Q Network(DQN)for air combat maneuvering decision-making.Considering the continuity between adjacent situations,the method takes multiple consecutive situations as one input of the neural network.To reflect the difference between adjacent situations,the method takes the difference of situation evaluation value as the reward of reinforcement learning.In different scenarios,the algorithm proposed in this paper is compared with the algorithm based on the Fully Neural Network(FNN)and the algorithm based on statistical principles respectively.The results show that,compared with the FNN algorithm,the algorithm proposed in this paper is more accurate and forwardlooking.Compared with the algorithm based on the statistical principles,the decision-making of the algorithm proposed in this paper is more efficient and its real-time performance is better.

Trajectory and Correction Maneuver During the Transfer from Earth to Halo Orbit

This article addresses the design of the trajectory transferring from Earth to Halo orbit, and proposes a timing closed-loop strategy of correction maneuver during the transfer in the frame of circular restricted three body problem （CR3BP）. The relation between the Floquet multipliers and the magnitudes of Halo orbit is established, so that the suitable magnitude for the aerospace mission is chosen in terms of the stability of Halo orbit. The stable manifold is investigated from the Poincar6 mapping defined which is different from the previous researches, and six types of single-impulse transfer trajectories are attained from the geometry of the invariant manifolds. Based on one of the trajectories of indirect transfer which are ignored in the most of literatures, the stochastic control theory for imperfect information of the discrete linear stochastic system is applied to design the trajectory correction maneuver. The statistical dispersion analysis is performed by Monte-Carlo simulation,

Fish maneuver analysis system based on sequence images

The aim is to establish an automatic system to analyze the maneuver performance of fish. A high speed camera （1 000 frame/s） is employed to record fast-start maneuver. Three steps are taken to analyze the kinematics： first, the midline in the first image is partitioned into equal interval lengths and the coordinates of all inter segmental points are saved. Secondly, these points coordinates are searched in the next frame with the digital image correlation （DIC） method, then these points are fitted with a spline curve function. Repeat this step until all the midlines are figured out frame by frame. Finally, according to the variety of midlines, the kinematics of the fast-start is calculated. Using this system to test carp C-start, the duration is divided into two stages： stage 1 is defined as the formation of the C shape and stage 2 as the return flip of the tail followed with forward motion. By tracing the middle line, the kinematic parameters of turning rate, centre of mass （CM） turning rate, CM turning radius, etc. are obtained.

An Effective Multiple Model Least Squares Method in Tracking of a Maneuvering Target

A polynomial model, time origin shifting model(TOSM, is used to describe the trajectory of a moving target .Based on TOSM, a recursive laeast squares(RLS) algorithm with varied forgetting factor is derived for tracking of a non-maneuvering target. In order to apply this algorithm to maneuvering targets tracking ,a tracking signal is performed on-line to determine what kind of TOSm will be in effect to track a target with different dynamics. An effective multiple model least squares filtering and forecasting method dadpted to real tracking of a maneuvering target is formulated. The algorithm is computationally more effcient than Kalman filter and the percentage improvement from simulations show both of them are considerably alike to some extent.

An approach to improving maneuver performance of coning algorithm

Aiming to improve the maneuver performance of the strapdown inertial navigation attitude coning algorithm a new coning correction structure is constructed by adding a sample to the traditional compressed coning correction structure. According to the given definition of classical coning motion the residual coning correction error based on the new coning correction structure is derived. On the basis of the new structure the frequency Taylor series method is used for designing a coning correction structure coefficient and then a new coning algorithm is obtained.Two types of error models are defined for the coning algorithm performance evaluation under coning environments and maneuver environments respectively.Simulation results indicate that the maneuver accuracy of the new 4-sample coning algorithm is almost double that of the traditional compressed 4-sample coning algorithm. The new coning algorithm has an improved maneuver performance while maintaining coning performance compared to the traditional compressed coning algorithm.

Maneuvering target tracking using threshold interacting multiple model algorithm

To avoid missing track caused by the target maneuvers in automatic target tracking system, a new maneuvering target tracking technique called threshold interacting multiple model （TIMM） is proposed. This algorithm is based on the interacting multiple model （IMM） method and applies a threshold controller to improve tracking accuracy. It is also applicable to other advanced algorithms of IMM. In this research, we also compare the position and velocity root mean square （RMS） errors of TIMM and IMM algorithms with two different examples. Simulation results show that the TIMM algorithm is superior to the traditional IMM alzorithm in estimation accuracy.

Autonomous air combat maneuver decision using Bayesian inference and moving horizon optimization

To reach a higher level of autonomy for unmanned combat aerial vehicle(UCAV) in air combat games, this paper builds an autonomous maneuver decision system. In this system,the air combat game is regarded as a Markov process, so that the air combat situation can be effectively calculated via Bayesian inference theory. According to the situation assessment result,adaptively adjusts the weights of maneuver decision factors, which makes the objective function more reasonable and ensures the superiority situation for UCAV. As the air combat game is characterized by highly dynamic and a significant amount of uncertainty,to enhance the robustness and effectiveness of maneuver decision results, fuzzy logic is used to build the functions of four maneuver decision factors. Accuracy prediction of opponent aircraft is also essential to ensure making a good decision; therefore, a prediction model of opponent aircraft is designed based on the elementary maneuver method. Finally, the moving horizon optimization strategy is used to effectively model the whole air combat maneuver decision process. Various simulations are performed on typical scenario test and close-in dogfight, the results sufficiently demonstrate the superiority of the designed maneuver decision method.

Erythropoietin ameliorates early ischemia-reperfusion injury following the Pringle maneuver

AIM:To investigate the protective effect of erythropoietin (Epo) against ischemia-reperfusion injury (IR/I) following the Pringle maneuver (PM),in comparison with conventional steroid administration in a prospective randomized trial. METHODS:Patients were randomized by age, sex, diagnosis, and surgical method, and assigned to three groups:(1) A steroid group (STRD, n= 9) who received 100 mg of hydrocortisone before PM, and on postoperative days 1, 2 and 3, followed by tapering until postoperative day 7; (2) An EPO1 group (n=10) who received 30 000 U of Epo before the PM and at the end of surgery; and (3) An EPO2 group (n=8) who received 60 000 U of Epo before the PM. Hemoglobin (Hb), hematocrit (Ht), aspartate aminotransferase (AST), alanine transaminase (ALT),lactate dehydrogenase (LDH), lactate, interleukin-6 (IL-6),and tumor necrosis factor(TNF)-α were measured before and just after (Day 0) surgery, and on postoperative days 1, 3, 7 and 14. RESULTS: There were no increases in Hb and Ht in the EPO1 and EPO2 groups. AST was signif icantly lower in EPO1 than in STRD on Day 0 (P=0.041), and lower in EPO1 than in STRD and EPO2 on Day 1 (P=0.018). ALT was signif icantly lower in EPO1 than in STRD and EPO2 on Day 0 (P=0.020) and Day 1 (P=0.004). There were no signif icant inter-group differences in the levels of LDH and lactate. IL-6 was signif icantly lower in EPO1 than in STRD and EPO2 on Day 0 (P=0.0036) and Day 1 (P=0.0451). TNF-α was signif icantly lower in EPO1 than in STRD and EPO2 on Day 0 (P=0.0006) and Day 1 (P<0.0001). Furthermore, hospitalization was signif icantly shorter in EPO1 and EPO2 than in STRD.CONCLUSION:Epo has greater potential than steroids to ameliorate IR/I after the PM. Epo at a dose of 30000 U, administered before PM and just after surgery, yields better results.