Recorded recurrent deep reinforcement learning guidance laws for intercepting endoatmospheric maneuvering missiles

This work proposes a recorded recurrent twin delayed deep deterministic(RRTD3)policy gradient algorithm to solve the challenge of constructing guidance laws for intercepting endoatmospheric maneuvering missiles with uncertainties and observation noise.The attack-defense engagement scenario is modeled as a partially observable Markov decision process(POMDP).Given the benefits of recurrent neural networks(RNNs)in processing sequence information,an RNN layer is incorporated into the agent’s policy network to alleviate the bottleneck of traditional deep reinforcement learning methods while dealing with POMDPs.The measurements from the interceptor’s seeker during each guidance cycle are combined into one sequence as the input to the policy network since the detection frequency of an interceptor is usually higher than its guidance frequency.During training,the hidden states of the RNN layer in the policy network are recorded to overcome the partially observable problem that this RNN layer causes inside the agent.The training curves show that the proposed RRTD3 successfully enhances data efficiency,training speed,and training stability.The test results confirm the advantages of the RRTD3-based guidance laws over some conventional guidance laws.

Design and Simulation of Controller for Hyper Velocity Kinetic Energy Missile

The controller design and digital simulation for the hyper velocity kinetic energy missile is investigated. A mathematical model of the trajectory deviation from the line of sight was established, the guidance closed loop was compensated with a phase advance lag corrective network, a selecting algorithm of the attitude control motors used to steer the missile's attitude was presented. In the presence of a wide variety of disturbances the results of digital simulation are satisfactory to circular error probability(CEP) being less than 0 5?m. The steering scheme utilizing attitude control motors as actuators to control the attitude of the missile is feasible.

3D-Route Planning Method to Realize Missile Cruise Flight

In order to increase the aircraft＇s survival in the flight mission, it is necessary to carry out flight mission planning, which includes TF/TA2（ terrain following/terrain avoidance/threat avoidance）. An approach to 3D-route planning based on A ＊ heuristic search algorithm was selected to determine the routes of fiber optic guidance missile＇ s cruise segment. The cost function was discussed, which was mainly related to the physical obstacle, threat exposure, and aircraft performance characteristics. The digital map techniques were presented, which included setting ＂no-go＂ area according to fiber＇s safety requirement. The optimal or the sub-optimal route was obtained, while the cost function constraints were satisfied and the stored terrain obtained from a real terrain was digitized. The algorithm is validated through simulation and can fulfill the route planning task which focuses on the cruise segment of fiber optic guidance missile.

Application of PID controller to 2D differential geometric guidance problem

This paper presents the application of the proportional-integral-derivative （PID） controller to the flight control system （FCS） for two-dimensional （2D） differential geometric （DG） guidance and control problem. In particular, the performance of the designed FCS is investigated. To this end, the commanded angle-of-attack is firstly developed in the time domain using the classical DG formulations. Then, the classical PID controller is introduced to develop a FCS so as to form the 2D DG guidance and control system, and the PID controller parameters are determined by the Ziegler-Nichols method as well as the Routh-Hurwitz stability algorithm to guarantee the convergence of the system error. The results demonstrate that the designed controller yields a fast responding system, and the resulting DG guidance and control system is viable and effective in a realistic missile defense engagement.

Differential geometric guidance command with finite time convergence using extended state observer

For improving the performance of differential geometric guidance command(DGGC), a new formation of this guidance law is proposed, which can guarantee the finite time convergence(FTC) of the line of sight(LOS) rate to zero or its neighborhood against maneuvering targets in three-dimensional(3D) space. The extended state observer(ESO) is employed to estimate the target acceleration, which makes the new DGGC more applicable to practical interception scenarios. Finally, the effectiveness of this newly proposed guidance command is demonstrated by the numerical simulation results.

Extended differential geometric guidance law for intercepting maneuvering targets

Without assumptions made on motion states of missile and target, an extended differential geometric guidance law is derived. Through introducing a line of sight rotation coordinate system, the derivation is simplified and has more explicit physical significances. The extended law is theoretically applicable to any engagement scenarios. Then, on basis of the extended law, a modified one is designed without the requirement of target acceleration and an approach is proposed to determining the applied direction of commanded missile acceleration. Qualitative analysis is carried out to study the capture performance and a criterion for capture is given. Simulation results indicate the two laws are effective and make up the deficiency that pure proportional navigation suitable for endoatmospheric interceptions cannot deal with high-speed maneuvering targets. Furthermore, the correctness of the criterion is validated.

Design of Closed Loop Optimal Guidance Law Using Neural Networks

It is generally impossible to obtain the analytic optimal guidance law for complex nonlinear guidance systems of homing missiles,and the open loop optimal guidance law is often obtained by numerical methods,which can not be used directly in practice.The neural networks are trained off line using the optimal trajectory of the missile produced by the numerical open loop optimal guidance law,and then,the converged neural networks are used on line as the feedback optimal guidance law in real time.The research shows that different selections of the neural networks inputs,such as the system state variables or the rate of LOS(line of sight),may have great effect on the performances of the guidance systems for homing missiles.The robustness for several guidance laws is investigated by simulations,and the modular neural networks architectures are used to increase the approximating and generalizing abilities in the large state space.Some useful conclusions are obtained by simulation results.

A hybrid proportional navigation based two-stage impact time control guidance law

To improve applicability and adaptability of the impact time control guidance(ITCG) in practical engineering, a twostage ITCG law with simple but effective structure is proposed based on the hybrid proportional navigation, namely, the pureproportional-navigation and the retro-proportional-navigation.For the case with the impact time error less than zero, the first stage of the guided trajectory is driven by the retro-proportionalnavigation and the second one is driven by the pure-proportionalnavigation. When the impact time error is greater than zero, both of the stages are generated by the pure-proportional-navigation but using different navigation gains. It is demonstrated by twoand three-dimensional numerical simulations that the proposed guidance law at least has comparable results to existing proportional-navigation-based ITCG laws and is shown to be advantageous in certain circumstances in that the proposed guidance law alleviates its dependence on the time-to-go estimation, consumes less control energy, and adapts itself to more boundary conditions and constraints. The results of this research are expected to be supplementary to the current research literature.

ROBUST GUIDANCE LAW FOR MANEUVERING TARGETS

A terminal guidance law is presented without any specific assumption on target maneuvering. A set of planar kinematical equations for a missile pursuing a target is derived first, followed by a brief analysis of proportional navigation (PN) guidance law. The robust guidance problem is formed after an analysis of guidance system, which is then reformed as the standard H ∞ control problem. A robust guidance law is finally deduced from H ∞ control theory. Simulations indicate that the missile employing this guidance law can attack more maneuverable targets than that with PN guidance law.

A distributed cooperative guidance law for salvo attack of multiple anti-ship missiles

The consensus problem of impact time is addressed for multiple anti-ship missiles. A new distributed cooperative guidance law with the form of biased proportional navigation guidance （BPNG） is presented. The proposed guidance law employs the available measurements of relative impact time error as the feedback information to achieve the consensus of impact time among mis- siles and, by exploiting the special structure of the biased cooperative control term, it can handle the seeker＇s field-of-view （FOV） constraint. The proposed scheme ensures convergence to consensus of impact time under either fixed or switching sensing/communication network, and the topological requirements are less restrictive than those in the existing results. Numerical examples are provided to illustrate the effectiveness of the proposed guidance law.

Integrated cooperative guidance framework and cooperative guidance law for multi-missile

An integrated cooperative guidance framework for multi-missile cooperatively attacking a single stationary target is proposed in this paper by combining both the centralized and decentralized communication topologies. Once missiles are distributed into several groups, missiles within a single group communicate with the centralized leader-follower framework, while the leaders from different groups communicate using the nearest-neighbor topology. To implement the integrated cooperative guidance framework, a group of Finite-Time Cooperative Guidance（FTCG） laws considering the saturation constraint on FOV（FTCG-FOV） are firstly derived within the centralized leader-follower framework to satisfy the communication topology of missiles in a single group.Then, an improved sequential approach is developed to adapt the FTCG-FOV to satisfy the communication topology between groups. The numerical simulations demonstrate the effectiveness and high efficiency of the integrated cooperative guidance framework and the cooperative guidance laws, as well as the superiority of the developed sequential approach.

A joint mid-course and terminal course cooperative guidance law for multi-missile salvo attack

Salvo attacking a surface target by multiple missiles is an effective tactic to enhance the lethality and penetrate the defense system. However, existing cooperative guidance laws in the midcourse or terminal course are not suitable for long-and medium-range missiles or stand-off attacking. Because the initial conditions of cooperative terminal guidance that are generally generated from the mid-course flight may not lead to a successful cooperative terminal guidance without proper mid-course flight adjustment. Meanwhile, cooperative guidance in the mid-course cannot solely guarantee the accuracy of a simultaneous arrival of multiple missiles. Therefore, a joint mid-course and terminal course cooperative guidance law is developed. By building a distinct leader-follower framework, this paper proposes an efficient coordinated Dubins path planning method to synchronize the arrival time of all engaged missiles in the mid-course flight. The planned flight can generate proper initial conditions for cooperative terminal guidance, and also benefit an earliest simultaneous arrival. In the terminal course, an existing cooperative proportional navigation guidance law guides all the engaged missiles to arrive at a target accurately and simultaneously.The integrated guidance law for an intuitive application is summarized. Simulations demonstrate that the proposed method can generate fast and accurate salvo attack.

Distributed cooperative guidance for multiple missiles with fixed and switching communication topologies

This study investigates cooperative guidance problems for multiple missiles with fixed and switching directed communication topologies. A two-step guidance strategy is proposed to realize the simultaneous attack. In the first step, a distributed cooperative guidance law is designed using local neighboring information for multiple missiles to achieve consensus on range-to-go and leading angle. The second step begins when the consensus of multiple missiles is realized. During the second step, multiple missiles disconnect from each other and hit the target using the proportional navigation guidance law. First, based on the local neighboring communications, a sufficient condition for multiple missiles to realize simultaneous attack with a fixed communication topology is presented, where the topology is only required to have a directed spanning tree. Then,the results are extended to the case of switching communication topologies. Finally, numerical simulations are provided to validate the theoretical results.

Saturated super-twisting sliding mode missile guidance

This paper concentrates on developing a missile terminal guidance law against a highly maneuvering target whose maneuvering acceleration is very close to that of the missile or even exceeds the missile normal acceleration in a finite period of time.A new saturated super-twisting algorithm is proposed and applied to the design of missile guidance law.The proposed algorithm has the advantages of simple structure,easy parameter tuning rules and a full utilization of the limit control input.The designed saturated super-twisting sliding mode guidance law is then employed in a missile guidance system.Simulation and its superior performance against strong maneuvering targets is demonstrated.

Three-dimensional cooperative guidance laws against stationary and maneuvering targets

This paper presents the cooperative strategies for salvo attack of multiple missiles based on the classical proportional navigation（PN） algorithm.The three-dimensional（3-D） guidance laws are developed in a quite simple formulation that consists of a PN component for target capture and a coordination component for simultaneous arrival.The centralized algorithms come into effect when the global information of time-to-go estimation is obtained, whereas the decentralized algorithms have better performance when each missile can only collect information from neighbors.Numerical simulations demonstrate that the proposed coordination algorithms are feasible to perform the cooperative engagement of multiple missiles against both stationary and maneuvering targets.The effectiveness of the 3-D guidance laws is also discussed.

Obstacle avoidance for multi-missile network via distributed coordination algorithm

A distributed coordination algorithm is proposed to enhance the engagement of the multi-missile network in consideration of obstacle avoidance. To achieve a cooperative interception, the guidance law is developed in a simple form that consists of three individual components for tar- get capture, time coordination and obstacle avoidance. The distributed coordination algorithm enables a group of interceptor missiles to reach the target simultaneously, even if some member in the multi-missile network can only collect the information from nearest neighbors. The simula- tion results show that the guidance strategy provides a feasible tool to implement obstacle avoid- ance for the multi-missile network with satisfactory accuracy of target capture. The effects of the gain parameters are also discussed to evaluate the proposed approach.

An adaptive fast fixed-time guidance law with an impact angle constraint for intercepting maneuvering targets

Sliding mode guidance laws based on a conventional terminal sliding mode guarantees only finite-time convergence, which verifies that the settling time is required to be estimated by selecting appropriate initial launched conditions. However, rapid convergence to a desired impact angle within a uniform bounded finite time is important in most practical guidance applications. A uniformly finite-time/fixed-time convergent guidance law means that the convergence（settling） time is predefined independently on initial conditions, that is, a closed-loop convergence time can be estimated a priori by guidance parameters. In this paper, a novel adaptive fast fixed-time sliding mode guidance law to intercept maneuver targets at a desired impact angle from any initial heading angle,with no problems of singularity and chattering, is designed. The proposed guidance law achieves system stabilization within bounded settling time independent on initial conditions and achieves more rapid convergence than those of fixed-time stable control methods by accelerating the convergence rate when the system is close to the origin. The achieved acceleration-magnitude constraints are rigorously enforced, and the chattering-free property is guaranteed by adaptive switching gains.Extensive numerical simulations are presented to validate the efficiency and superiority of the proposed guidance law for different initial engagement geometries and impact angles.

Optimal three-dimensional impact time guidance with seeker’s field-of-view constraint

This paper proposes a new three-dimensional optimal guidance law for impact time control with seeker’s Field-of-View(FOV) constraint to intercept a stationary target. The proposed guidance law is devised in conjunction with the concept of biased Proportional Navigation Guidance(PNG). The guidance law developed leverages a nonlinear function to ensure the boundedness of velocity lead angle to cater to the seeker’s FOV limit. It is proven that the impact time error is nullified in a finite-time under the proposed method. Additionally, the optimality of the biased command is theoretically analyzed. Numerical simulations confirm the superiority of the proposed method and validate the analytic findings.

Distributed three-dimensional cooperative guidance via receding horizon control

The paper presents a new three-dimensional (3D) cooperative guidance approach by the receding horizon control (RHC) technique. The objective is to coordinate the impact time of a group of interceptor missiles against the stationary target. The framework of a distributed RHC scheme is developed, in which each interceptor missile is assigned its own finite-horizon optimal control problem (FHOCP) and only shares the information with its neighbors. The solution of the local FHOCP is obtained by the constrained particle swarm optimization (PSO) method that is integrated into the distributed RHC framework with enhanced equality and inequality constraints. The numerical simulations show that the proposed guidance approach is feasible to implement the cooperative engagement with satisfied accuracy of target capture. Finally, the computation efficiency of the distributed RHC scheme is discussed in consideration of the PSO parameters, control update period and prediction horizon. (C) 2016 Chinese Society of Aeronautics and Astronautics. Production and hosting by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license.

Guidance law with impact time and impact angle constraints

A novel closed-form guidance law with impact time and impact angle constraints is pro- posed for salvo attack of anti-ship missiles, which employs missile＇s normal acceleration （not jerk） as the control command directly. Firstly, the impact time control problem is formulated as tracking the designated time-to-go （the difference between the designated impact time and the current flight time） for the actual time-to-go of missile, and the impact angle control problem is formulated as tracking the designated heading angle for the actual heading angle of missile. Secondly, a biased proportional navigation guidance （BPNG） law with designated heading angle constraint is constructed, and the actual time-to-go estimation for this BPNG is derived analytically by solving the system differential equations. Thirdly, by adding a feedback control to this constructed BPNG to eliminate the time-to-go errorthe difference between the standard time-to-go and the actual time-to-go, a guidance law with adjustable coefficients to control the impact time and impact angle simultaneously is developed. Finally, simulation results demonstrate the performance and feasibility of the proposed approach.