基于强化学习的微小型弹药两回路驾驶仪设计

Design Two-loop Autopilot Based on Reinforcement Learning for Miniature Munition

利用强化学习及自适应动态规划原理,设计了一种适用于微小型弹药的两回路驾驶仪,并建立了纵向通道控制模型。由于跟踪器问题的最优解不易获得,将系统矩阵与期望输出信号进行增广,构成增广系统并引入折扣因子,将系统的跟踪器设计问题转换为调节器设计问题。基于贝尔曼最优性原理,采用策略迭代的方法对黎卡提方程进行求解,并证明了该算法的收敛性。最后仿真验证了通过策略评估及策略更新两步迭代计算,可以收敛至跟踪器的最优解。

Based on theory of reinforcement learning and adaptive dynamic programming,the two-loop autopilot of miniature munition is designed.Besides,a linear miniature munition control model of longitudinal channel is established.Because it is difficult to get the optimal solution of tracker,the argument system is constructed by putting the system sate matrix and expected signal together,and the tracker problem is transformed into a regulator problem by augmenting the system and introducing a discount factor.Based on the Bellman optimality principle,the iterative is used to solve the Riccati equation,and the convergence of the iterative algorithm is proved.The simulation results show the tracker can converge to optimal solution through the iteration by two steps of strategy evaluation and strategy updating.