Multi-UAV cooperative maneuver decision-making for pursuitevasion using improved MADRL

Aiming at the problem of multi-UAV pursuit-evasion confrontation, a UAV cooperative maneuver method based on an improved multi-agent deep reinforcement learning(MADRL) is proposed. In this method, an improved Comm Net network based on a communication mechanism is introduced into a deep reinforcement learning algorithm to solve the multi-agent problem. A layer of gated recurrent unit(GRU) is added to the actor-network structure to remember historical environmental states. Subsequently,another GRU is designed as a communication channel in the Comm Net core network layer to refine communication information between UAVs. Finally, the simulation results of the algorithm in two sets of scenarios are given, and the results show that the method has good effectiveness and applicability.

UAV maneuvering decision-making algorithm based on deep reinforcement learning under the guidance of expert experience

Autonomous umanned aerial vehicle(UAV) manipulation is necessary for the defense department to execute tactical missions given by commanders in the future unmanned battlefield. A large amount of research has been devoted to improving the autonomous decision-making ability of UAV in an interactive environment, where finding the optimal maneuvering decisionmaking policy became one of the key issues for enabling the intelligence of UAV. In this paper, we propose a maneuvering decision-making algorithm for autonomous air-delivery based on deep reinforcement learning under the guidance of expert experience. Specifically, we refine the guidance towards area and guidance towards specific point tasks for the air-delivery process based on the traditional air-to-surface fire control methods.Moreover, we construct the UAV maneuvering decision-making model based on Markov decision processes(MDPs). Specifically, we present a reward shaping method for the guidance towards area and guidance towards specific point tasks using potential-based function and expert-guided advice. The proposed algorithm could accelerate the convergence of the maneuvering decision-making policy and increase the stability of the policy in terms of the output during the later stage of training process. The effectiveness of the proposed maneuvering decision-making policy is illustrated by the curves of training parameters and extensive experimental results for testing the trained policy.

Situational continuity-based air combat autonomous maneuvering decision-making

In order to improve the performance of UAV's autonomous maneuvering decision-making,this paper proposes a decision-making method based on situational continuity.The algorithm in this paper designs a situation evaluation function with strong guidance,then trains the Long Short-Term Memory(LSTM)under the framework of Deep Q Network(DQN)for air combat maneuvering decision-making.Considering the continuity between adjacent situations,the method takes multiple consecutive situations as one input of the neural network.To reflect the difference between adjacent situations,the method takes the difference of situation evaluation value as the reward of reinforcement learning.In different scenarios,the algorithm proposed in this paper is compared with the algorithm based on the Fully Neural Network(FNN)and the algorithm based on statistical principles respectively.The results show that,compared with the FNN algorithm,the algorithm proposed in this paper is more accurate and forwardlooking.Compared with the algorithm based on the statistical principles,the decision-making of the algorithm proposed in this paper is more efficient and its real-time performance is better.

Evolutionary Decision-Making and Planning for Autonomous Driving Based on Safe and Rational Exploration and Exploitation

Decision-making and motion planning are extremely important in autonomous driving to ensure safe driving in a real-world environment.This study proposes an online evolutionary decision-making and motion planning framework for autonomous driving based on a hybrid data-and model-driven method.First,a data-driven decision-making module based on deep reinforcement learning(DRL)is developed to pursue a rational driving performance as much as possible.Then,model predictive control(MPC)is employed to execute both longitudinal and lateral motion planning tasks.Multiple constraints are defined according to the vehicle’s physical limit to meet the driving task requirements.Finally,two principles of safety and rationality for the self-evolution of autonomous driving are proposed.A motion envelope is established and embedded into a rational exploration and exploitation scheme,which filters out unreasonable experiences by masking unsafe actions so as to collect high-quality training data for the DRL agent.Experiments with a high-fidelity vehicle model and MATLAB/Simulink co-simulation environment are conducted,and the results show that the proposed online-evolution framework is able to generate safer,more rational,and more efficient driving action in a real-world environment.

UAV Maneuvering Decision-Making Algorithm Based on Twin Delayed Deep Deterministic Policy Gradient Algorithm

Aiming at intelligent decision-making of unmanned aerial vehicle(UAV)based on situation information in air combat,a novelmaneuvering decision method based on deep reinforcement learning is proposed in this paper.The autonomous maneuvering model ofUAV is established byMarkovDecision Process.The Twin DelayedDeep Deterministic Policy Gradient(TD3)algorithm and the Deep Deterministic Policy Gradient(DDPG)algorithm in deep reinforcement learning are used to train the model,and the experimental results of the two algorithms are analyzed and compared.The simulation experiment results show that compared with the DDPG algorithm,the TD3 algorithm has stronger decision-making performance and faster convergence speed and is more suitable for solving combat problems.The algorithm proposed in this paper enables UAVs to autonomously make maneuvering decisions based on situation information such as position,speed,and relative azimuth,adjust their actions to approach,and successfully strike the enemy,providing a new method for UAVs to make intelligent maneuvering decisions during air combat.

Is the feminization of labor harmful to agricultural production? The decision-making and production control perspective

Even today,academics continue to debate the effect of feminization of agricultural labor force on agricultural output.By considering the dimensions of participation in decision-making and production,this study divides the various agricultural production models into three types:(i)the traditional model of decisions made either jointly by men and women or by men alone while both genders participate in production,(ii)complete feminization of agricultural decision-making and the production labor force,and(iii)feminization of the agricultural production labor force only.This study investigates the effects of combining or separating decision-making and production in regard to agricultural development in the context of feminization of the agricultural labor force.Using follow-up data collected from 2004–2008 by the Ministry of Agriculture of China,we built a comprehensive panel data model to test our hypotheses.Our research shows that in comparison to traditional agricultural households and fully feminized agricultural labor forces,partially feminized production resulted in lower grain yield and technological advancement.The feminization of agricultural labor does not necessarily have a negative impact on agricultural output,especially since heavy manual labor is being increasingly replaced by agricultural machinery and outsourcing of tasks.The degree of feminization of the decision-making and production processes should be an important consideration when evaluating the purported negative effects of the feminization of agricultural labor.

Decision-Making in Driver-Automation Shared Control:A Review and Perspectives

Shared control schemes allow a human driver to work with an automated driving agent in driver-vehicle systems while retaining the driver’s abilities to control.The human driver,as an essential agent in the driver-vehicle shared control systems,should be precisely modeled regarding their cognitive processes,control strategies,and decision-making processes.The interactive strategy design between drivers and automated driving agents brings an excellent challenge for human-centric driver assistance systems due to the inherent characteristics of humans.Many open-ended questions arise,such as what proper role of human drivers should act in a shared control scheme?How to make an intelligent decision capable of balancing the benefits of agents in shared control systems?Due to the advent of these attentions and questions,it is desirable to present a survey on the decision making between human drivers and highly automated vehicles,to understand their architectures,human driver modeling,and interaction strategies under the driver-vehicle shared schemes.Finally,we give a further discussion on the key future challenges and opportunities.They are likely to shape new potential research directions.

Adaptive variable structure control based on backstepping for spacecraft with reaction wheels during attitude maneuver

An adaptive variable structure control method based on backstepping is proposed for the attitude maneuver problem of rigid spacecraft with reaction wheel dynamics in the presence of uncertain inertia matrix and external disturbances. The proposed control approach is a combination of the backstepping and the adaptive variable structure control. The cascaded structure of the attitude maneuver control system with reaction wheel dynamics gives the advantage for applying the backstepping method to construct Lyapunov functions. The robust stability to external disturbances and parametric uncertainty is guaranteed by the adaptive variable structure control. To validate the proposed control algorithm, numerical simulations using the proposed approach are performed for the attitude maneuver mission of rigid spacecraft with a configuration consisting of four reaction wheels for actuator and three magnetorquers for momentum unloading. Simulation results verify the effectiveness of the proposed control algorithm.

Improved filtered-ε adaptive inverse control and its application on nonlinear ship maneuvering

The drawbacks of common nonlinear Filtered-ε adaptive inverse control （AIC） method, such as the unreliability due to the change of delay time and the faultiness existing in its disturbance control loop, are discussed. Based on it, the diagram of AIC is amended to accommodate with the characteristic of nonlinear object with time delay. The corresponding Filtered-ε adaptive algorithm based on RTRL is presented to identify the parameters and design the controller. The simulation results on a nonlinear ship model of ＂The R.O.V Zeefakker＂ show that compared with the previous scheme and adaptive PID control, the improved method not only keeps the same dynamic response performance, but also owns higher robustness and disturbance rejection ability, and it is suitable for the control of nonlinear objects which have higher requirement to the maneuverability under complex disturbance environment.

Consensus intuitionistic fuzzy group decision-making method for aircraft cockpit display and control system evaluation

A novel group decision-making （GDM） method based on intuitionistic fuzzy sets （IFSs） is developed to evaluate the ergonomics of aircraft cockpit display and control system （ACDCS）. The GDM process with four steps is discussed. Firstly, approaches are proposed to transform four types of common judgement representations into a unified expression by the form of the IFS, and the features of unifications are analyzed. Then, the aggregation operator called the IFSs weighted averaging （IFSWA） operator is taken to synthesize decision-makers’ （DMs’） preferences by the form of the IFS. In this operator, the DM’s reliability weights factors are determined based on the distance measure between their preferences. Finally, an improved score function is used to rank alternatives and to get the best one. An illustrative example proves the proposed method is effective to valuate the ergonomics of the ACDCS.

Enhancing Safety in Autonomous Vehicle Navigation:An Optimized Path Planning Approach Leveraging Model Predictive Control

This paper explores the application of Model Predictive Control(MPC)to enhance safety and efficiency in autonomous vehicle(AV)navigation through optimized path planning.The evolution of AV technology has progressed rapidly,moving from basic driver-assistance systems(Level 1)to fully autonomous capabilities(Level 5).Central to this advancement are two key functionalities:Lane-Change Maneuvers(LCM)and Adaptive Cruise Control(ACC).In this study,a detailed simulation environment is created to replicate the road network between Nantun andWuri on National Freeway No.1 in Taiwan.The MPC controller is deployed to optimize vehicle trajectories,ensuring safe and efficient navigation.Simulated onboard sensors,including vehicle cameras and millimeterwave radar,are used to detect and respond to dynamic changes in the surrounding environment,enabling real-time decision-making for LCM and ACC.The simulation resultshighlight the superiority of the MPC-based approach in maintaining safe distances,executing controlled lane changes,and optimizing fuel efficiency.Specifically,the MPC controller effectively manages collision avoidance,reduces travel time,and contributes to smoother traffic flow compared to traditional path planning methods.These findings underscore the potential of MPC to enhance the reliability and safety of autonomous driving in complex traffic scenarios.Future research will focus on validating these results through real-world testing,addressing computational challenges for real-time implementation,and exploring the adaptability of MPC under various environmental conditions.This study provides a significant step towards achieving safer and more efficient autonomous vehicle navigation,paving the way for broader adoption of MPC in AV systems.

Variable structure attitude maneuver and vibration control of flexible spacecraft

A dual-stage control system design method is presented for the three-axis-rotational maneuver and vibration stabilization of a spacecraft with flexible appendages embedded with piezoceramics as sensor and actuator. In this design approach, the attitude control and the vibration suppression sub-systems are designed separately using the lower order model. The design of attitude controller is based on the variable structure control (VSC) theory leading to a discontinuous control law. This controller accomplishes asymptotic attitude maneuvering in the closed-loop system and is insensitive to the interaction of elastic modes and uncertainty in the system. To actively suppress the flexible vibrations, the modal velocity feedback control method is presented by using piezoelectric materials as additional sensor and actuator bonded on the surface of the flexible appendages. In addition, a special configuration of actuators for three-axis attitude control is also investigated: the pitch attitude controlled by a momentum wheel, and the roll/yaw control achieved by on-off thrusters, which is modulated by pulse width pulse frequency modulation technique to construct the proper control torque history. Numerical simulations performed show that the rotational maneuver and vibration suppression are accomplished in spite of the presence of disturbance torque and parameter uncertainty.

Application of Fuzzy Automata Decision-Making System in Target Control

In order to perform better in target control, this paper proposed a decision-making system method based on fuzzy automata. The decision-making system first preprocessed the signal and then performed a two-level decision on the target to achieve optimal control. The system consisted of four parts: signal preprocessing, contrast decision-making, comprehensive judgment of decision-making and decision-making result. These decision algorithms in target control were given. A concrete application of this decision-making system in target control was described. Being compared with other existing methods, this paper used both global features and local features of target, and used the decision-making system of fuzzy automata for the target control. Simulation results showed that the control effect based on the decision-making system was better than that of the other existing methods. Not only it was faster, but also its correct control rate was higher to be 95.18% for the target control. This research on the control system not only developed the FA theory, but also strengthened its application scope in the field of control engineering.

Special issue on Networked sensing,decision-making and control

Networked sensing and control has attracted significant interest in recent years due to its wide applications. For example, sensor networks, especially wireless sensor networks, have found important applications in environmental monitoring, agriculture, building and industrial automation, machine condition monitoring, intelligent transportation systems, health care, surveillance, and defense. On the other hand, due to the flexibility and significant COSt-saving,

Trajectory and Correction Maneuver During the Transfer from Earth to Halo Orbit

This article addresses the design of the trajectory transferring from Earth to Halo orbit, and proposes a timing closed-loop strategy of correction maneuver during the transfer in the frame of circular restricted three body problem （CR3BP）. The relation between the Floquet multipliers and the magnitudes of Halo orbit is established, so that the suitable magnitude for the aerospace mission is chosen in terms of the stability of Halo orbit. The stable manifold is investigated from the Poincar6 mapping defined which is different from the previous researches, and six types of single-impulse transfer trajectories are attained from the geometry of the invariant manifolds. Based on one of the trajectories of indirect transfer which are ignored in the most of literatures, the stochastic control theory for imperfect information of the discrete linear stochastic system is applied to design the trajectory correction maneuver. The statistical dispersion analysis is performed by Monte-Carlo simulation,

Research on Maneuver Decision-Making of Multi-Agent Adversarial Game in a Random Interference Environment

The strategy evolution process of game players is highly uncertain due to random emergent situations and other external disturbances.This paper investigates the issue of strategy interaction and behavioral decision-making among game players in simulated confrontation scenarios within a random interference environment.It considers the possible risks that random disturbances may pose to the autonomous decision-making of game players,as well as the impact of participants’manipulative behaviors on the state changes of the players.A nonlinear mathematical model is established to describe the strategy decision-making process of the participants in this scenario.Subsequently,the strategy selection interaction relationship,strategy evolution stability,and dynamic decision-making process of the game players are investigated and verified by simulation experiments.The results show that maneuver-related parameters and random environmental interference factors have different effects on the selection and evolutionary speed of the agent’s strategies.Especially in a highly uncertain environment,even small information asymmetry or miscalculation may have a significant impact on decision-making.This also confirms the feasibility and effectiveness of the method proposed in the paper,which can better explain the behavioral decision-making process of the agent in the interaction process.This study provides feasibility analysis ideas and theoretical references for improving multi-agent interactive decision-making and the interpretability of the game system model.

Variable structure guidance law for attacking surface maneuver targets

The characteristics of surface maneuver targets are analyzed and a 3-D relative motion model for missiles and targets is established. A variable structure guidance law is designed considering the characteristics of targets. In the guidance law, the distance between missiles and targets as well as the missile-target relative velocity are all substituted by estimation values. The estimation errors, the target＇s velocity, and the maneuver acceleration are all treated as bounded disturbance. The guidance law proposed can be implemented conveniently in engineering with little target information. The performance of the guidance system is analyzed theoretically and the numerical simulation result shows the effectiveness of the guidance law.

UAV cooperative air combat maneuver decision based on multi-agent reinforcement learning

In order to improve the autonomous ability of unmanned aerial vehicles(UAV)to implement air combat mission,many artificial intelligence-based autonomous air combat maneuver decision-making studies have been carried out,but these studies are often aimed at individual decision-making in 1 v1 scenarios which rarely happen in actual air combat.Based on the research of the 1 v1 autonomous air combat maneuver decision,this paper builds a multi-UAV cooperative air combat maneuver decision model based on multi-agent reinforcement learning.Firstly,a bidirectional recurrent neural network(BRNN)is used to achieve communication between UAV individuals,and the multi-UAV cooperative air combat maneuver decision model under the actor-critic architecture is established.Secondly,through combining with target allocation and air combat situation assessment,the tactical goal of the formation is merged with the reinforcement learning goal of every UAV,and a cooperative tactical maneuver policy is generated.The simulation results prove that the multi-UAV cooperative air combat maneuver decision model established in this paper can obtain the cooperative maneuver policy through reinforcement learning,the cooperative maneuver policy can guide UAVs to obtain the overall situational advantage and defeat the opponents under tactical cooperation.

Chaotic attitude and reorientation maneuver for completely liquid-filled spacecraft with flexible appendage

The present paper investigates the chaotic attitude dynamics and reorientation maneuver for completely viscous liquid-filled spacecraft with flexible appendage. All of the equations of motion are derived by using Lagrangian mechanics and then transformed into a form consisting of an unperturbed part plus perturbed terms so that the system＇s nonlinear characteristics can be exploited in phase space. Emphases are laid on the chaotic attitude dynamics produced from certain sets of physical parameter values of the spacecraft when energy dissipation acts to derive the body from minor to major axis spin. Numerical solutions of these equations show that the attitude dynamics of liquid-filled flexible spacecraft possesses characteristics common to random, non- periodic solutions and chaos, and it is demonstrated that the desired reorientation maneuver is guaranteed by using a pair of thruster impulses. The control strategy for reorientation maneuver is designed and the numerical simulation results are presented for both the uncontrolled and controlled spins transition.

Nonlinear differential geometric guidance for maneuvering target

Based on the idea of zeroing the line of sight rate（LOSR）,a novel nonlinear differential geometric（DG） law for intercepting the agile target is proposed.In the first part,the DG formulations are utilized to describe the relatively kinematics model of missile and target,and the nonlinear DG guidance（DGG） law is proposed based on the nonlinear control theory to eliminate the influence brought by target.Further,the missile guidance commands are derived to overcome the information loss caused by decoupling condition,the new necessary initial condition is developed to guarantee capture the agile target.Then,the designed nonlinear DGG commands are transformed from an arc-length system to the time domain.A desirable aspect of the designed guidance law is that it does not require rigorous information about target acceleration.Representative numerical results show that the designed guidance law obtain a better performance than the traditional DGG law for agile target.