Computational intelligence interception guidance law using online off-policy integral reinforcement learning

Missile interception problem can be regarded as a two-person zero-sum differential games problem,which depends on the solution of Hamilton-Jacobi-Isaacs(HJI)equa-tion.It has been proved impossible to obtain a closed-form solu-tion due to the nonlinearity of HJI equation,and many iterative algorithms are proposed to solve the HJI equation.Simultane-ous policy updating algorithm(SPUA)is an effective algorithm for solving HJI equation,but it is an on-policy integral reinforce-ment learning(IRL).For online implementation of SPUA,the dis-turbance signals need to be adjustable,which is unrealistic.In this paper,an off-policy IRL algorithm based on SPUA is pro-posed without making use of any knowledge of the systems dynamics.Then,a neural-network based online adaptive critic implementation scheme of the off-policy IRL algorithm is pre-sented.Based on the online off-policy IRL method,a computa-tional intelligence interception guidance(CIIG)law is developed for intercepting high-maneuvering target.As a model-free method,intercepting targets can be achieved through measur-ing system data online.The effectiveness of the CIIG is verified through two missile and target engagement scenarios.

Differential game strategy in three-player evasion and pursuit scenarios

A conflict of three players, including an attacker, a defender, and a target with bounded control is discussed based on the differential game theories in which the target and the defender use an optimal pursuit strategy. The current approach chooses the miss distance as the outcome of the conflict. Different optimal guidance laws are investigated, and feasible conditions are analyzed for the attacker to accomplish an attacking task. For some given conditions, the attacker cannot intercept the target by only using a one-to-one optimal pursuit guidance law; thus, a guidance law for the attacker to reach a critical safe value is investigated.Specifically, the guidance law is divided into two parts. Before the engagement time between the defender and the attacker, the attacker uses this derived guidance law to guarantee that the evasion distance from the defender is safe, and that the zero-effort-miss(ZEM) distance between the attacker and the target is the smallest.After that engagement time, the attacker uses the optimal one-toone guidance law to accomplish the pursuit task. The advantages and limited conditions of these derived guidance laws are also investigated by using nonlinear simulations.

Three-dimensional nonlinear H_2/H_∞ guidance law based upon approach of solving the state feedback Nash balance point

Based upon the theory of the nonlinear quadric two-person nonzero-sum differential game,the fact that the time-limited mixed H2/H∞ control problem can be turned into the problem of solving the state feedback Nash balance point is mentioned. Upon this,a theorem about the solution of the state feedback control is given,the Lyapunov stabilization of the nonlinear system under this control is proved,too. At the same time,this solution is used to design the nonlinear H2/H∞ guidance law of the relative motion between the missile and the target in three-dimensional(3D) space. By solving two coupled Hamilton-Jacobi partial differential inequalities(HJPDI),a control with more robust stabilities and more robust performances is obtained. With different H∞ performance indexes,the correlative weighting factors of the control are analytically designed. At last,simulations under different robust performance indexes and under different initial conditions and under the cases of intercepting different maneuvering targets are carried out. All results indicate that the designed law is valid.

An Adaptive Weighted Differential Game Guidance Law

For intercepting modern high maneuverable targets, a novel adaptive weighted differential game guidance law based on the game theory of mixed strategy is proposed, combining two guidance laws which are derived from the perfect and imperfect in- formation pattern, respectively. The weights vary according to the estimated error of the target＇s acceleration, the guidance law is generated by directly using the estimation of target＇s acceleration when the estimated error is small, and a differential game guidance law with adaptive penalty coefficient is implemented when the estimated error is large. The adaptive penalty coeffi- cients are not constants and they can be adjusted with current target maneuverability. The superior homing performance of the new guidance law is verified by computer simulations.

基于博弈论的弹目攻防决策方法研究

针对空战环境中弹目攻防双方的对抗特性,提出了一种基于博弈论的弹目攻防决策方法。基于导弹目标运动数学关系得到状态方程,根据弹目攻防对抗机理建立“一对一导弹-目标”动态博弈模型,确定弹目双方策略集与收益矩阵,提出混合策略纳什均衡求解方法,并结合模型滚动预测方法获得该策略空间的纳什均衡点。算例仿真结果表明,基于混合策略下导弹制导律为该策略空间的纳什均衡点,且该方法可以减小导弹对目标的脱靶量,提高导弹的命中精度,为导弹攻防作战提供了依据。

Optimal guidance against active defense ballistic missiles via differential game strategies

The optimal guidance problem for an interceptor against a ballistic missile with active defense is investigated in this paper.A class of optimal guidance schemes are proposed based on linear quadratic differential game method and numerical solution of Riccati differential equation.By choosing proper parameters, the proposed guidance schemes are able to drive the interceptor to the target and away from the defender simultaneously.Additionally, fuel cost, control saturation,chattering phenomenon and parameters selection were taken into account.Satisfaction of the proposed guidance schemes of the saddle point condition is proven theoretically.Finally, nonlinear numerical examples are included to demonstrate the effectiveness and performance of the developed guidance approaches.Comparison of control performance between different guidance schemes are presented and analysis.

基于DGL/IMM算法的随机机动弹头拦截研究

讨论了反导拦截高速随机机动再入弹头弹道末段的制导律和滤波器设计。基于线化假设和完全信息对策模型,推导出微分对策制导律(DGL)的表达式。在此基础上考虑一般拦截情形,即非完全信息对策以及测量含噪声,提出采用交互式多模型滤波(IMM)估计状态变量,推导出滤波方程,并与以往方法进行比较,仿真分析表明所提出的算法滤波效果好,运算量低,脱靶量均值小,而且对于未知目标机动模式有很强的适应能力。

纯方位量测下增强可观测性的微分对策制导律

为改善导弹受到干扰或加装被动捷联导引头仅能够完成视线角度或纯方位量测情况下的导弹制导信息估计性能,提出了一种适用于纯方位量测的微分对策制导律。首先,通过对范数型微分对策制导律对策空间的分析,明确了捕获区的制导特性可以用于改进纯方位测量情况下的可观测性的设计思路。其次,在完成纯方位估计器设计的基础上,利用估计均方误差反映估计器可观测性的特性,建立了导弹控制量和可观测性之间的关系。然后在微分对策空间的捕获区域内,进行制导控制指令的选取,保证脱靶量的同时,提升估计器的可观测性能。最后给出了所提出的增强可观测性的制导律的执行流程,并考虑控制系统延迟和外部其他不确定性因素的影响,以及脱靶量控制优先的原则,对零控脱靶量参数和剩余飞行时间参数进行了设计。制导律的设计方法超出了确定性等价原理的框架,实现了滤波和制导律的集成设计。非线性仿真对比表明,采用所提出的制导律,可以获得较好的目标信息估计性能,且具有较高的单发命中概率。

基于微分对策的多飞行器协同捕获空间划分

针对多飞行器协同围捕问题,提出一种基于微分对策的加权制导律下捕获空间划分方法。选取合适的性能函数,建立多对一协同围捕微分对策模型;利用哈密尔顿函数法求解得出飞行器和目标的最优控制策略,获得各飞行器的零控脱靶量的最优轨迹;以2个飞行器为例,设置1个2对1协同围捕场景,根据2个飞行器初始零控脱靶量的符号和大小关系,给出各飞行器单独拦截以及协同围捕状态下的相关定义;根据各飞行器零控脱靶量的最优轨迹,围绕初始零控脱靶量给出1种协同捕获空间的划分方法;设置3个具体的仿真场景,验证结果证明了该方法的准确性。

Guidance laws for attacking defended target

In this paper,two new guidance laws based on differential game theory are proposed and investigated for the attacker in an attacker-defender-target scenario.The conditions for the attacker winning the game are analyzed when the target and defender using the differential game guidance law based on the linear model.The core ideas underlying the two guidance laws are the attacker evading to a critical safe boundary from the defender,and then maintaining a critical miss distance.The guidance law more appropriate for the attacker to win the game differs according to the initial parameters.Unlike other guidance laws,when using the derived guidance laws there is no need to know the target and the defender’s control efforts.The results of numerical simulations show that the attacker can evade the defender and hit the target successfully by using the proposed derived guidance laws.

基于全局信息的非线性微分对策协同制导律

为了适应未来战争中多弹攻防作战的强博弈对抗场景,研究了基于全局信息的非线性微分对策协同制导律。考虑由我方多枚导弹攻击某一空中目标,目标释放多枚拦截弹拦截我方导弹的场景,假设导弹之间能够互相通信,建立了对应的非线性微分对策模型,并利用模型预测方法进行滚动时域PSO算法求解该非线性微分对策问题,得到相应的控制策略,改进了建立在小角度线性化假设基础上的传统线性微分对策不适用于多弹间强博弈场景的问题。仿真结果表示非线性微分对策制导律能够完成躲避敌方多枚导弹拦截的前提下命中目标的任务。

微分对策制导律研究的现状及展望

本文介绍了微分对策制导律（ＤＧＧＬ）研究的现状，对其定性分析方法，尤其是定量分析方法进行了回顾和评价，提出了ＤＧＧＬ未来要研究的重点以及对这种制导律的一些展望。

基于模糊理论的微分对策制导律

在现代战场环境下，导弹制导所需信息经常受到各种干扰，有时甚至是不确定的。而基于精确模型的微分对策制导律的实时实现面临着许多困难。为解决这个问题，尝试将模糊理论与微分对策制导律相结合，采用多级模糊控制规则，以导弹与飞机格斗为例,给出了模型的建立方法。并探讨了这种模糊微分对策制导律实现的可行性和优点，为将来发展基于微分对策理论的智能制导律提供了一定的思路。

制导规律研究现状及展望

系统地论述了古典和现代的一些制导规律的研究现状 ,分析了它们的优点和不足、适应范围和局限性 ,并讨论了它们的发展前景。其中着重论述了比例导引、最优导引、微分对策和近年来热门研究的神经网络控制。可供从事导弹制导的科研人员对现有制导规律有一个全面的认识 ,以便于更深入地研究。

具有终端约束的线性二次型微分对策制导律

主要针对具有终端碰撞角度约束的线性二次型微分对策制导问题进行研究。首先,采用终端投影法定义零效脱靶量和零效碰撞角作为新的状态变量,实现了系统降阶;然后,基于零和微分对策原理,进行制导律的推导,并对导航增益、鞍点解的存在域和理想拦截性能进行了分析;最后,分别针对目标作常值机动和方波机动2种情况对系统性能进行了仿真。结果表明,当导弹具有足够的机动能力时,可满足较小的脱靶量和碰撞角误差要求,从而改善导弹杀伤性能。

攻击主动防御飞行器的微分对策制导律

基于微分对策理论设计了躲避护卫弹的同时攻击飞行器的制导律。根据传统的性能指标推导了该场景的微分对策制导律,并且根据权重系数的取值定义了三种制导律:最优追赶制导律、逃脱-追赶制导律和复合制导律。最优追赶制导律容易被护卫弹拦截,逃脱-追赶制导律容易造成导弹和飞行器的零控脱靶量急剧增大而使得攻击失败,复合制导律很难选择合适的权重系数。针对以上不足,提出了两种改进的制导律,并对该两种制导律的适用情况进行了分析。通过非线性模型仿真,验证了这两种方法的可行性。该两种制导律目的性强,攻击导弹可以躲避护卫弹进而攻击飞行器。

有界双重控制导弹微分对策制导律

针对有界控制导弹采用鸭舵或尾舵单一控制形式存在的劣势,基于双边优化微分对策理论,推导了一种有界双重控制导弹微分对策制导律。该制导律不仅将鸭舵与尾舵两组舵面的控制有效融合在一起,而且实现了有界控制命令最优的分配设计。分析了该微分对策制导律的对策空间,并从弹目机动性能比和控制系统时间常数比之间的关系,给出了鞍点解的存在条件。考虑非完全信息情形,完成了目标加速度滤波器和拦截性能衡量指标的设计。采用Monte Carlo法进行了制导性能的仿真验证,结果表明:所设计的有界双重控制导弹制导律与采用单一的鸭舵控制或尾舵控制的导弹相比不仅机动性要求较低,且具有较高的命中概率。

寻的导弹范数型微分对策制导律

为改进寻的导弹拦截性能,基于微分对策理论推导了一种一般形式的范数型微分对策制导律(NDGL),该制导律可适用于拦截导弹和目标具有任意阶控制的情形。针对该制导律的三种典型形式及其对策空间进行了分析和研究,并提出了NDGL的一种延迟补偿改进形式。该改进NDGL利用目标姿态信息对加速度不确定集进行实时调整,进一步削弱了估计延迟对系统性能的影响。分别对完全信息情形下的一般形式NDGL和非完全信息情形下的四种典型NDGL的性能进行了仿真研究和比较,结果验证了该一般形式NDGL,表明了所给出的改进NDGL的拦截性能优势。

针对高速机动目标的三维非线性微分对策制导律

为有效拦截高超声速飞行器,提出了一种基于预测控制的三维微分对策制导律。该制导律将高速机动目标拦截末制导过程抽象为以视线角速率和燃料消耗为性能指标的零和微分对策问题,以非线性模型预测控制为框架,采用梯度下降法实时求解决策约束型有限时域微分对策,得到最优决策。该制导律能够在满足动态系统高维非线性、机动能力和燃料消耗等要求下,实现对目标的精确拦截;在与比例导引律燃料消耗相当的情况下,具有末端弹道平直、制导精度高等优点。

基于神经网络和微分对策理论的制导律

根据微分对策制导律目前面临的困境，本文尝试将神经网络引入微分对策制导律。采用ＢＰ模型，将微分对策的定量和定性分析结合起来。以导弹与飞机格斗为例，探讨其实现的可能性和优点，为将来发展基于微分对策理论的智能制导律提供了一定的思路。