Recorded recurrent deep reinforcement learning guidance laws for intercepting endoatmospheric maneuvering missiles

This work proposes a recorded recurrent twin delayed deep deterministic(RRTD3)policy gradient algorithm to solve the challenge of constructing guidance laws for intercepting endoatmospheric maneuvering missiles with uncertainties and observation noise.The attack-defense engagement scenario is modeled as a partially observable Markov decision process(POMDP).Given the benefits of recurrent neural networks(RNNs)in processing sequence information,an RNN layer is incorporated into the agent’s policy network to alleviate the bottleneck of traditional deep reinforcement learning methods while dealing with POMDPs.The measurements from the interceptor’s seeker during each guidance cycle are combined into one sequence as the input to the policy network since the detection frequency of an interceptor is usually higher than its guidance frequency.During training,the hidden states of the RNN layer in the policy network are recorded to overcome the partially observable problem that this RNN layer causes inside the agent.The training curves show that the proposed RRTD3 successfully enhances data efficiency,training speed,and training stability.The test results confirm the advantages of the RRTD3-based guidance laws over some conventional guidance laws.

Saturated super-twisting sliding mode missile guidance

This paper concentrates on developing a missile terminal guidance law against a highly maneuvering target whose maneuvering acceleration is very close to that of the missile or even exceeds the missile normal acceleration in a finite period of time.A new saturated super-twisting algorithm is proposed and applied to the design of missile guidance law.The proposed algorithm has the advantages of simple structure,easy parameter tuning rules and a full utilization of the limit control input.The designed saturated super-twisting sliding mode guidance law is then employed in a missile guidance system.Simulation and its superior performance against strong maneuvering targets is demonstrated.

Application of PID controller to 2D differential geometric guidance problem

This paper presents the application of the proportional-integral-derivative （PID） controller to the flight control system （FCS） for two-dimensional （2D） differential geometric （DG） guidance and control problem. In particular, the performance of the designed FCS is investigated. To this end, the commanded angle-of-attack is firstly developed in the time domain using the classical DG formulations. Then, the classical PID controller is introduced to develop a FCS so as to form the 2D DG guidance and control system, and the PID controller parameters are determined by the Ziegler-Nichols method as well as the Routh-Hurwitz stability algorithm to guarantee the convergence of the system error. The results demonstrate that the designed controller yields a fast responding system, and the resulting DG guidance and control system is viable and effective in a realistic missile defense engagement.

Extended differential geometric guidance law for intercepting maneuvering targets

Without assumptions made on motion states of missile and target, an extended differential geometric guidance law is derived. Through introducing a line of sight rotation coordinate system, the derivation is simplified and has more explicit physical significances. The extended law is theoretically applicable to any engagement scenarios. Then, on basis of the extended law, a modified one is designed without the requirement of target acceleration and an approach is proposed to determining the applied direction of commanded missile acceleration. Qualitative analysis is carried out to study the capture performance and a criterion for capture is given. Simulation results indicate the two laws are effective and make up the deficiency that pure proportional navigation suitable for endoatmospheric interceptions cannot deal with high-speed maneuvering targets. Furthermore, the correctness of the criterion is validated.

A hybrid proportional navigation based two-stage impact time control guidance law

To improve applicability and adaptability of the impact time control guidance(ITCG) in practical engineering, a twostage ITCG law with simple but effective structure is proposed based on the hybrid proportional navigation, namely, the pureproportional-navigation and the retro-proportional-navigation.For the case with the impact time error less than zero, the first stage of the guided trajectory is driven by the retro-proportionalnavigation and the second one is driven by the pure-proportionalnavigation. When the impact time error is greater than zero, both of the stages are generated by the pure-proportional-navigation but using different navigation gains. It is demonstrated by twoand three-dimensional numerical simulations that the proposed guidance law at least has comparable results to existing proportional-navigation-based ITCG laws and is shown to be advantageous in certain circumstances in that the proposed guidance law alleviates its dependence on the time-to-go estimation, consumes less control energy, and adapts itself to more boundary conditions and constraints. The results of this research are expected to be supplementary to the current research literature.

ROBUST GUIDANCE LAW FOR MANEUVERING TARGETS

A terminal guidance law is presented without any specific assumption on target maneuvering. A set of planar kinematical equations for a missile pursuing a target is derived first, followed by a brief analysis of proportional navigation (PN) guidance law. The robust guidance problem is formed after an analysis of guidance system, which is then reformed as the standard H ∞ control problem. A robust guidance law is finally deduced from H ∞ control theory. Simulations indicate that the missile employing this guidance law can attack more maneuverable targets than that with PN guidance law.

3D-Route Planning Method to Realize Missile Cruise Flight

In order to increase the aircraft＇s survival in the flight mission, it is necessary to carry out flight mission planning, which includes TF/TA2（ terrain following/terrain avoidance/threat avoidance）. An approach to 3D-route planning based on A ＊ heuristic search algorithm was selected to determine the routes of fiber optic guidance missile＇ s cruise segment. The cost function was discussed, which was mainly related to the physical obstacle, threat exposure, and aircraft performance characteristics. The digital map techniques were presented, which included setting ＂no-go＂ area according to fiber＇s safety requirement. The optimal or the sub-optimal route was obtained, while the cost function constraints were satisfied and the stored terrain obtained from a real terrain was digitized. The algorithm is validated through simulation and can fulfill the route planning task which focuses on the cruise segment of fiber optic guidance missile.

A distributed cooperative guidance law for salvo attack of multiple anti-ship missiles

The consensus problem of impact time is addressed for multiple anti-ship missiles. A new distributed cooperative guidance law with the form of biased proportional navigation guidance （BPNG） is presented. The proposed guidance law employs the available measurements of relative impact time error as the feedback information to achieve the consensus of impact time among mis- siles and, by exploiting the special structure of the biased cooperative control term, it can handle the seeker＇s field-of-view （FOV） constraint. The proposed scheme ensures convergence to consensus of impact time under either fixed or switching sensing/communication network, and the topological requirements are less restrictive than those in the existing results. Numerical examples are provided to illustrate the effectiveness of the proposed guidance law.

Integrated cooperative guidance framework and cooperative guidance law for multi-missile

An integrated cooperative guidance framework for multi-missile cooperatively attacking a single stationary target is proposed in this paper by combining both the centralized and decentralized communication topologies. Once missiles are distributed into several groups, missiles within a single group communicate with the centralized leader-follower framework, while the leaders from different groups communicate using the nearest-neighbor topology. To implement the integrated cooperative guidance framework, a group of Finite-Time Cooperative Guidance（FTCG） laws considering the saturation constraint on FOV（FTCG-FOV） are firstly derived within the centralized leader-follower framework to satisfy the communication topology of missiles in a single group.Then, an improved sequential approach is developed to adapt the FTCG-FOV to satisfy the communication topology between groups. The numerical simulations demonstrate the effectiveness and high efficiency of the integrated cooperative guidance framework and the cooperative guidance laws, as well as the superiority of the developed sequential approach.

A joint mid-course and terminal course cooperative guidance law for multi-missile salvo attack

Salvo attacking a surface target by multiple missiles is an effective tactic to enhance the lethality and penetrate the defense system. However, existing cooperative guidance laws in the midcourse or terminal course are not suitable for long-and medium-range missiles or stand-off attacking. Because the initial conditions of cooperative terminal guidance that are generally generated from the mid-course flight may not lead to a successful cooperative terminal guidance without proper mid-course flight adjustment. Meanwhile, cooperative guidance in the mid-course cannot solely guarantee the accuracy of a simultaneous arrival of multiple missiles. Therefore, a joint mid-course and terminal course cooperative guidance law is developed. By building a distinct leader-follower framework, this paper proposes an efficient coordinated Dubins path planning method to synchronize the arrival time of all engaged missiles in the mid-course flight. The planned flight can generate proper initial conditions for cooperative terminal guidance, and also benefit an earliest simultaneous arrival. In the terminal course, an existing cooperative proportional navigation guidance law guides all the engaged missiles to arrive at a target accurately and simultaneously.The integrated guidance law for an intuitive application is summarized. Simulations demonstrate that the proposed method can generate fast and accurate salvo attack.

Distributed cooperative guidance for multiple missiles with fixed and switching communication topologies

This study investigates cooperative guidance problems for multiple missiles with fixed and switching directed communication topologies. A two-step guidance strategy is proposed to realize the simultaneous attack. In the first step, a distributed cooperative guidance law is designed using local neighboring information for multiple missiles to achieve consensus on range-to-go and leading angle. The second step begins when the consensus of multiple missiles is realized. During the second step, multiple missiles disconnect from each other and hit the target using the proportional navigation guidance law. First, based on the local neighboring communications, a sufficient condition for multiple missiles to realize simultaneous attack with a fixed communication topology is presented, where the topology is only required to have a directed spanning tree. Then,the results are extended to the case of switching communication topologies. Finally, numerical simulations are provided to validate the theoretical results.

Three-dimensional cooperative guidance laws against stationary and maneuvering targets

This paper presents the cooperative strategies for salvo attack of multiple missiles based on the classical proportional navigation（PN） algorithm.The three-dimensional（3-D） guidance laws are developed in a quite simple formulation that consists of a PN component for target capture and a coordination component for simultaneous arrival.The centralized algorithms come into effect when the global information of time-to-go estimation is obtained, whereas the decentralized algorithms have better performance when each missile can only collect information from neighbors.Numerical simulations demonstrate that the proposed coordination algorithms are feasible to perform the cooperative engagement of multiple missiles against both stationary and maneuvering targets.The effectiveness of the 3-D guidance laws is also discussed.

Obstacle avoidance for multi-missile network via distributed coordination algorithm

A distributed coordination algorithm is proposed to enhance the engagement of the multi-missile network in consideration of obstacle avoidance. To achieve a cooperative interception, the guidance law is developed in a simple form that consists of three individual components for tar- get capture, time coordination and obstacle avoidance. The distributed coordination algorithm enables a group of interceptor missiles to reach the target simultaneously, even if some member in the multi-missile network can only collect the information from nearest neighbors. The simula- tion results show that the guidance strategy provides a feasible tool to implement obstacle avoid- ance for the multi-missile network with satisfactory accuracy of target capture. The effects of the gain parameters are also discussed to evaluate the proposed approach.

An adaptive fast fixed-time guidance law with an impact angle constraint for intercepting maneuvering targets

Sliding mode guidance laws based on a conventional terminal sliding mode guarantees only finite-time convergence, which verifies that the settling time is required to be estimated by selecting appropriate initial launched conditions. However, rapid convergence to a desired impact angle within a uniform bounded finite time is important in most practical guidance applications. A uniformly finite-time/fixed-time convergent guidance law means that the convergence（settling） time is predefined independently on initial conditions, that is, a closed-loop convergence time can be estimated a priori by guidance parameters. In this paper, a novel adaptive fast fixed-time sliding mode guidance law to intercept maneuver targets at a desired impact angle from any initial heading angle,with no problems of singularity and chattering, is designed. The proposed guidance law achieves system stabilization within bounded settling time independent on initial conditions and achieves more rapid convergence than those of fixed-time stable control methods by accelerating the convergence rate when the system is close to the origin. The achieved acceleration-magnitude constraints are rigorously enforced, and the chattering-free property is guaranteed by adaptive switching gains.Extensive numerical simulations are presented to validate the efficiency and superiority of the proposed guidance law for different initial engagement geometries and impact angles.

Optimal three-dimensional impact time guidance with seeker’s field-of-view constraint

This paper proposes a new three-dimensional optimal guidance law for impact time control with seeker’s Field-of-View(FOV) constraint to intercept a stationary target. The proposed guidance law is devised in conjunction with the concept of biased Proportional Navigation Guidance(PNG). The guidance law developed leverages a nonlinear function to ensure the boundedness of velocity lead angle to cater to the seeker’s FOV limit. It is proven that the impact time error is nullified in a finite-time under the proposed method. Additionally, the optimality of the biased command is theoretically analyzed. Numerical simulations confirm the superiority of the proposed method and validate the analytic findings.

Composite impact vector control based on Apollo descent guidance

A new three-dimensional missile guidance law to control the impact vector against a stationary target is proposed.The composite guidance law has two well-known components:Apollo descent guidance and trajectory shaping guidance.These respectively linear and planar guidance laws are combined to achieve a specified impact direction.The main idea is to define an impact plane and to steer the missile onto this plane using Apollo descent guidance while concurrently performing trajectory shaping with reference to the impact plane.The resulting guidance law is expressed by a single equation in vector form,which is straightforward to implement.Because it originates from an optimal formulation,the performance of the proposed technique is expected to be satisfactory.This is confirmed by comparative simulation runs,which also involve the method known as generalized explicit guidance.

Guidance law with impact time and impact angle constraints

A novel closed-form guidance law with impact time and impact angle constraints is pro- posed for salvo attack of anti-ship missiles, which employs missile＇s normal acceleration （not jerk） as the control command directly. Firstly, the impact time control problem is formulated as tracking the designated time-to-go （the difference between the designated impact time and the current flight time） for the actual time-to-go of missile, and the impact angle control problem is formulated as tracking the designated heading angle for the actual heading angle of missile. Secondly, a biased proportional navigation guidance （BPNG） law with designated heading angle constraint is constructed, and the actual time-to-go estimation for this BPNG is derived analytically by solving the system differential equations. Thirdly, by adding a feedback control to this constructed BPNG to eliminate the time-to-go errorthe difference between the standard time-to-go and the actual time-to-go, a guidance law with adjustable coefficients to control the impact time and impact angle simultaneously is developed. Finally, simulation results demonstrate the performance and feasibility of the proposed approach.

Optimal guidance of extended trajectory shaping

To control missile＇s miss distance as well as terminal impact angle, by involving the timeto-go-nth power in the cost function, an extended optimal guidance law against a constant maneuvering target or a stationary target is proposed using the linear quadratic optimal control theory.An extended trajectory shaping guidance（ETSG） law is then proposed under the assumption that the missile-target relative velocity is constant and the line of sight angle is small. For a lag-free ETSG system, closed-form solutions for the missile＇s acceleration command are derived by the method of Schwartz inequality and linear simulations are performed to verify the closed-form results. Normalized adjoint systems for miss distance and terminal impact angle error are presented independently for stationary targets and constant maneuvering targets, respectively. Detailed discussions about the terminal misses and impact angle errors induced by terminal impact angle constraint, initial heading error, seeker zero position errors and target maneuvering, are performed.

Ideal proportional navigation for exoatmospheric interception

Ideal proportional navigation （IPN） is a natural choice for exoatmospheric interception for its mighty capture capability and ease of implementation. The closed-form solution of two- dimensional ideal proportional navigation was conducted in previous public literature, whereas the practical interception happens in the three-dimensional space. A novel set of relative dynamic equations is adopted in this paper, which is with the advantage of decoupling relative motion in the instantaneous rotation plane of the line of sight from the rotation of this plane. The dimension-reduced IPN is constructed in this instantaneous plane, which functions as a three-dimensional guidance law. The trajectory features of dimension-reduced IPN are explored, and the capture regions of dimension-reduced IPN with limited acceleration against nonmaneuvering and maneuvering targets are analyzed by using phase plane method. It is proved that the capture capability of IPN is much stronger than true proportional navigation （TPN）, no matter the target maneuvers or not. Finally, simulation results indicate that IPN is more effective than TPN in exoatmospheric interception scenarios.

Linear Quadratic Differential Game Strategies with Two-pursuit Versus Single-evader

In order to intercept the future targets that are characterized by high maneuverability, multiple interceptors may be launched and aimed at single target. The scenario of two missiles P and Q intercepting a single target is modeled as a two-pursuit single-evader non-zero-sum linear quadratic differential game. The intercept space is decomposed into three subspaces which are mutually disjoint and their union covers the entire intercept space. The effect of adding the second interceptor arises in the intercept space of both P and Q （PQ-intercept space）. A guidance law is derived from the Nash equilibrium strategy set （NESS） of the game. Simulation studies are focused on the PQ-intercept space. It is indicated that 1） increasing the target＇s maneuverability will enlarge PQ-intercept space; 2） the handover conditions will be released if the initial zero-effort-miss （ZEM） of both interceptors has opposite sign; 3） overvaluation of the target＇s maneuverability by choosing a small weight coefficient will generate robust performance with respect to the target maneuvering command switch time and decrease the fuel requirement; and 4） cooperation between interceptors increases the interception probability.